Go: Enabling DevOps To Go Faster (w/ Notes)

Transcript

1. GO: ENABLING DEVOPS TO GO FASTER ORCHESTRUCTURE @ChrisShort devopsish.com ‣ Hello!
2. CREDITS LICENSE AND MATERIALS ▸ Gopher Artwork from Ashley McNamara: https://github.com/ashleymcnamara/gophers ▸ All product names, logos, and brands are property of their respective owners. All company, product and service names used in this work are for identification purposes only. Use of these names, logos, and brands does not imply endorsement. ▸ This presentation is licensed under the Creative Commons Attribution-ShareAlike 4.0 International license. ▸ You are encouraged to remix, transform, or build upon the material, providing you distribute your contributions under the same license. ▸ This presentation will be available on chrisshort.net on or after 31 Jan 2018. @ChrisShort devopsish.com
3. INTRODUCTION @ChrisShort devopsish.com chrisshort.net ‣ Hi I'm Chris Short ‣ I'm a Senior DevOps Advocate at SJ Technologies ‣ I write a weekly newsletter called DevOps'ish ‣ I was recently named a Cloud Native Computing Foundation Ambassador ‣ I'm co-lead of the DevOps community at opensource.com ‣ I'm co-lead of the Detroit Go Meetup ‣ And I have a few other things I do in my non-existent free time
4. I'M ALSO A GOPHER @ChrisShort devopsish.com Chris Short in Gopher Form by Gopherize.me ‣ I'm many things ‣ One thing I'm not is a coder ‣ But, Go is starting to change that
5. WHAT IS GO? GO: ENABLING DEVOPS TO GO FASTER @ChrisShort devopsish.com
6. WHAT IS GO? GOVERVIEW ▸ "Go is an open source programming language that makes it easy to build simple, reliable, and efficient software." ▸ Development started in 2007 ▸ Public release in 2009 ▸ Go 1.0 released in 2012 ▸ A lot of thought went into Go @ChrisShort devopsish.com ‣ "Go is an open source programming language that makes it easy to build simple, reliable, and efficient software." ‣ Development started in 2007 ‣ Publicly released in 2009 ‣ 1.0 in 2012 ‣ Go development is very thoughtful and methodical
7. WHAT IS GO? WHO MADE GO? ▸ Programming language created at ▸ Created by Robert Griesemer, Rob Pike, Ken Thompson ▸ Later adding Ian Lance Taylor and Russ Cox ▸ These cats have done some things: ▸ Sawzall (Hadoop), first window system for Unix in 1981, Google's V8 Engine, Plan 9 from Bell Labs, UTF-8, B programming language (C predecessor), regular expressions, GCC, the gold linker, and more @ChrisShort devopsish.com ‣ Created at Google ‣ A bunch of really smart people made Go ‣ Robert Griesemer ‣ Rob Pike ‣ Ken Thompson ‣ Ian Lance Taylor ‣ Russ Cox ‣ Some seriously brilliant minds
8. WHAT IS GO? WHY MAKE GO? ▸ "No new major systems language in a decade." —Rob Pike ▸ Designed with the following advances in technology in mind: ▸ Modern Networking ▸ Multi-core CPUs ▸ Slowing of Moore's Law ▸ Improved safety, high speed compilation, and communications @ChrisShort devopsish.com ‣ Think of the language landscape in 2009 ‣ What was the newest language? Python? PHP? ‣ These languages didn't handle modern computing concepts well
9. WHAT IS GO? GO VS. OTHER LANGUAGES @ChrisShort devopsish.com Go Others Clean/minimalist Java? No header files C/C++ Efficient Garbage Collection Fast compilation ‣ Go is clean and minimalist ‣ Makes it extremely easy to learn and use ‣ It's so easy this Ops turned DevOps clown can talk to you about it ‣ There aren't a mess of header files like C ‣ Garbage collection is quick and efficient ‣ Concurrency enables multiple functions to run simultaneously ‣ Go programs compile very quickly
10. WHAT IS GO? SORRY, DEVELOPERS @ChrisShort devopsish.com https://xkcd.com/303/ ‣ I fixed this comic for you ‣ Sorry, devs
11. WHAT IS GO? GO TOOLING ▸ Standard Library is AMAZING! ▸ Intuitive packages: ▸ fmt ▸ crypto ▸ log @ChrisShort devopsish.com ▸ net and net/http ▸ os ▸ syscall ‣ The standard library is really, really good. ‣ The packages are sane and intuitive ‣ fmt will literally format your code for you ‣ crypto is right in line with the RFC ‣ log allows you to exit the program with an exit code ‣ net package is very well written ‣ syscall and os just work
12. GO: STANDARD LIBRARY "IT JUST WORKS." Dave Chaney.... @ChrisShort devopsish.com ‣ According to Dave Chaney, "It just works."
13. WHAT IS GO? WHO CONTROLS GO? ▸ It's open source! The community! ▸ Go was developed at Google by Google Folks ▸ But, look who is writing Go code ▸ #2: Microsoft ▸ #4: Apache ▸ #6 Alibaba @ChrisShort devopsish.com ‣ Go was created at Google ‣ I know what you're thinking ‣ "I don't have Google's problems" ‣ You don't but you sure can benefit from their work ‣ Microsoft, Apache, and Alibaba are!
14. WHAT IS GO GOOD AT? GO: ENABLING DEVOPS TO GO FASTER @ChrisShort devopsish.com
15. @ChrisShort devopsish.com ‣ There are some obvious projects that use Go ‣ Let's not talk about the obvious stuff ‣ That's boring
16. WHAT IS GO GOOD AT? CONTAINER RUNTIMES ▸ Go is a lower-level language (like C and C++) ▸ Interacts with kernel directly; not through a VM (like Java) ▸ Go easily manages processes, syscalls, etc. ▸ Go's concurrency model makes for efficient core/thread use ▸ Multi-architecture builds ▸ Static compilation @ChrisShort devopsish.com ‣ Container runtimes are generally written in go these days ‣ Go is a lower-level language ‣ Go doesn't use a virtual machine ‣ Go's process handling and straightforward syscall interactions make for clean interactions with the system ‣ Look up Liz Rice's GopherCon 2017 talk ‣ She walks through building strace in 60 lines of Go ‣ Concurrency makes for great resource utilization ‣ You can compile for x86 as easily as you can arm ‣ Your output is a static binary ‣ Everything in one artifact ready for deployment
17. WHAT IS GO GOOD AT? CRYPTOCURRENCIES ▸ Ethereum has the #3 GitHub project for Go ▸ geth is the Go implementation of Ethereum client ▸ geth is the default Ethereum client ▸ geth became the "reference client" @ChrisShort devopsish.com ‣ geth is the go implementation of the Ethereum client ‣ It's also the default Ethereum client ‣ geth became the reference client for all other clients written in other languages
18. GO: CRYPTOCURRENCIES "THE TRUE POWER OF ... GO WAS THE EASE OF USE AND THE POWER OF COMMUNICATING CONCEPTS..." Jeffrey Wilcke... @ChrisShort devopsish.com ‣ Just read this quote. ‣ That's so amazing. ‣ It was easier to understand the Ethereum concepts laid out in the white paper in Go than any other language
19. WHAT IS GO GOOD AT? STORAGE SYSTEMS ▸ Dropbox's Magic Pocket is a multi-exabyte storage
 system written in (mostly) Go ▸ Rewrite of prototype was necessary ▸ Go addresses the need for massively distributed systems ▸ 100K LOC written by 4 people in only @ChrisShort devopsish.com ‣ When Dropbox pulled out of AWS they landed on Go ‣ Prototype that was written in Python ‣ After testing that wasn't going to scale ‣ Plus they wanted Go for it's type-safe and concurrency ‣ 4 people re-wrote the prototype in Go ‣ 100K LOC of Go code were hammered out in six months ‣ They did end up writing some bits in Rust too
20. WHAT IS GO GOOD AT? PROJECTS UTILIZING GO @ChrisShort devopsish.com ‣ This is just a snapshot of projects using Go extensively today
21. WHAT IS GO GOOD AT? CNCF LANDSCAPE (NOT ALL GO) @ChrisShort devopsish.com ‣ For those that haven't seen this before this is the Cloud Native Computing Foundation Landscape ‣ This is a collection of tools that handle Cloud Native concepts very well ‣ Quite a bit of the CNCF Landscape tooling is written in Go ‣ Not all of it is though
22. WHAT IS GO GOOD AT? GOPINIONS ▸ When asked, "Why does Go make you happy?" Go devs responded with: ▸ "Less is more." —Kris Nova, Heptio ▸ "Go does a really awesome job at making the easy things really easy, and the complicated things easy to understand while not abstracting them away."
 —Julia Ferraioli, Google ▸ "Go makes me happy because it's so cool it has its own set of proverbs! go- proverbs.github.io" —Carlisia Pinto, Fastly ▸ "Comprehensible parallelism that won't shoot you in the foot is Go's most winsome feature." —Liz Fong-Jones, Google Cloud @ChrisShort devopsish.com ‣ I asked some friends why Go makes them happy
23. HOW GO BAILED ME OUT GO: ENABLING DEVOPS TO GO FASTER @ChrisShort devopsish.com Let me tell you a story of how Go saved my ass once
24. NOT TOO LONG AGO IN A PLACE OF WORK FAR, FAR AWAY... @ChrisShort devopsish.com ‣ My team of merry DevOps'ers inherited an application ‣ A third-party built the app a few years ago ‣ The app had long been abandoned ‣ Before we could do any re-engineering work, we had to resolve a critical issue. ‣ The certificates were about to expire!
25. AND OF COURSE PRODUCTION @ChrisShort devopsish.com ‣ Oh! ‣ And the only environment this application was in was production ‣ And there was no time to implement a new key management system ‣ And it was a pet project of someone in senior leadership
26. LET'S TALK CERTIFICATE CHAINS @ChrisShort devopsish.com ‣ Let's Talk Certificate Chains for a minute ‣ HTTPS, SSL... it's all TLS now! ‣ In my opinion, if you're using TLS you MUST have a rock solid configuration ‣ This means you have to include the certificate chain in the correct order ‣ This is no longer optional in the post-Heartbleed world ‣ The Internets are watching and if they find something wrong with your setup they'll exploit it
27. THIS IS THE GOAL @ChrisShort devopsish.com ‣ This is the goal ‣ If you are going to bother to encrypt your traffic you better do it right ‣ This is what we're aiming for; an A+ ‣ At this company, we obtained certs from a preferred vendor that our company was cool with ‣ I prefer Let's Encrypt but some companies aren't comfortable with that yet for various reasons ‣ The process goes like this... ‣ You generate your CSR and private key ‣ You send the CSR to the vendor ‣ The certificate arrives but usually doesn't have an intermediate key in chain because... vendors be vendoring
28. ... @ChrisShort devopsish.com ‣ No big deal ‣ Let's go to the vendor's documentation... ‣ And OMG... ‣ The vendor docs are terrible ‣ This is when you learn... ‣ Cryptography is hard but implementing cryptographic best practices might be even harder
29. ‣ What do we do? ‣ Look at statistical probabilities and start shuffling keys around? ‣ The series of games you have to play with openssl or nginx or some other method aren't intuitive ‣ Do you know how hard this is to explain to people?
30. SO WHAT DOES ANY GOOD ENGINEER DO? @ChrisShort devopsish.com ‣ We needed a tool that would fail at the sign of an improper certificate chain ‣ We needed a lightweight tool that could be publicly accessible ‣ Conducting a third-party analysis of the certificates and configuration was also a requirement ‣ There were no tools that I could find meeting these needs ‣ I decided to build my own tool and I chose to build it with Go
31. THREE GO PACKAGES LOG ▸ The Go log package is pretty self explanatory ▸ Package that enables logging ▸ Needed a spectacular failure at the sign of trouble ▸ log has three helper functions: print, fatal, and panic @ChrisShort devopsish.com ‣ Let me just say the Go standard library is amazing! ‣ log is designed beautifully ‣ Using fatal to break the app and log to stderr if something isn't right is great!
32. THREE GO PACKAGES CRYPTO/TLS ▸ The Go crypto/tls package partially implements TLS 1.2, as specified in RFC-5246 ▸ Package configures usable SSL/TLS versions ▸ Identifies preferred cipher suites and elliptic curves used during handshakes ▸ This is the package that handles connections securely @ChrisShort devopsish.com ‣ The crypto/tls package is a splendid implementation of the RFC ‣ "It just works."
33. THREE GO PACKAGES NET/HTTP ▸ Go implementation of HTTP ▸ net/http has a function called ListenAndServeTLS ▸ ListenAndServeTLS provides the desired certificate checking functionality ▸ "If the certificate is signed by a certificate authority, the certFile should be the concatenation of the server's certificate, any intermediates, and the CA's certificate." @ChrisShort devopsish.com ‣ net/http has the ListenAndServeTLS function and it's awesome ‣ It fails if your certs aren't up to snuff or ordered properly ‣ It helps us Gophers out immensely by enforcing best practices
34. THREE GO PACKAGES MAIN: MUX, CFG, SRV ▸ Code creates a mux, short for HTTP request multiplexer ▸ I ❤ multiplexers (it's a long story that involves analog signals) ▸ mux has a function that creates an HTTP server with headers and content (Hello World!) ▸ cfg brings in all the TLS bits seen in a solid web server config ▸ srv puts the pieces together and defines what port to listen on @ChrisShort devopsish.com ‣ READ SLIDE
35. THREE GO PACKAGES FAIL SPECTACULARLY ▸ I ❤ DevOps and I embrace failure ▸ log.Fatal(srv.ListenAndServeTLS("/etc/ssl-tester/tls.crt", "/etc/ssl-tester/ tls.key")) ▸ Defines path of certificate files to use ▸ Logs a fatal error if certificate is not valid ▸ Fails Fast @ChrisShort devopsish.com ‣ I love DevOps and I embrace failure ‣ The code allows us to fail quickly if the certificates aren't in accordance with RFC ‣ Stuff in the standard library JUST WORKS
36. IT'S OPEN SOURCE! @ChrisShort devopsish.com https://github.com/chris-short/ssl-tester ‣ The code is open source ‣ Check it out on GitHub ‣ Throw a star my way if you feel like it
37. IT WORKS! @ChrisShort devopsish.com ‣ Yes it works! ‣ Yes you can access it right now! ‣ ssl-tester.chrisshort.net sends you to a reference implementation
38. NO. IT REALLY WORKS! @ChrisShort devopsish.com ‣ You can even scan it with external tools! ‣ There's the A+
39. HOW GO BAILED ME OUT RECAP ▸ The Go code does exactly what I need it to do ▸ About 40 lines of code!!! I ❤ Go! ▸ Binary is a self contained web server ▸ Compiles to less than 6MB!!! I ❤ Go! ▸ Can be safely deployed to any public server ▸ External testing run against it for extra vetting @ChrisShort devopsish.com ‣ The tool does exactly what I need it to do and nothing more ‣ It fails when the certificate chain provided is incorrect ‣ It's lightweight and publicly accessible ‣ I'm able to test via third-parties ‣ It's a tiny, single binary
40. @ChrisShort devopsish.com ‣ If you want to start learning go checkout tour.golang.org ‣ Or ping me for some book recommendations
41. GO: ENABLING DEVOPS TO GO FASTER CLEAR IS BETTER THAN CLEVER. Rob Pike—- @ChrisShort devopsish.com
42. CONCLUSION @ChrisShort devopsish.com