DevOpsDays Cuba 2017: DevOps Archeology

Author: Lee Fox
Summary: Back in at the 2001 OOPSLA Conference, the term Software Archeology was first coined. It gave a set of tools and processes for understanding a poorly documented and poorly maintained software architecture. It didn’t just work for the undocumented codebases, but ones that are incredibly large and complex, even if good documentation exists. The idea was to follow the true archeology with investigative work to understand the mindsets of the people who did the original implementations. Nowadays, the problem of understanding software still exists, but has also grown by a quite with the larger scales of production environments that are often times distributed or found in the cloud. So, this session will help the DevOps engineer walking into a new shop with tools and techniques to understand, not only the code, but the entire production system as well. This will help provide faster ramp time, higher quality software implementations, and more stable production environments.

Transcript

1. Lee Fox @FoxInATX 1
2. • Cloud Architect at Infor • Agile, DevOps, ChatOps evangelist • Involved with Agile Austin, Cloud Austin • Alexa Developer • Amateur Chef 2
3. • Java development background • AWS background 3
4. • DevOps Archeology • Dig Techniques 4
10. • First appeared at 2001 OOPSLA conference as a workshop • Gain perspective and understanding into application behavior • Understand what predecessors were thinking 10
11. • First appeared at Austin DevOps, October 2017 • Applying Concept to Infrastructure • Infrastructure as code • Learn what predecessors were thinking • Understand how systems work together 11
12. • This is about the artifacts—the technology • Culture and processes forthcoming in DevOps Anthropology 12
13. • Scripting languages • Ongoing documentation • Signature Analysis and software visualization • Reverse Engineering tools • Operating System level traces • Coupling Analysis • Source Control • Search Engines • IDE file browsing • Unit testing • API documentation generation tools • Debuggers • Dependency Analysis 13
14. • Monitoring tools • Log Aggregation • Automation tools • Testing Tools
17. • Start by observing the box • Describe it • Conduct experiments 17
18. • Isolate the artifact • Apply stimuli to the artifact • Measure the response of the artifact • Repeat as necessary 18
20. • Visualizations Provide easy ways to understand code • A picture is worth 1000 words • Visualizations are focused along a single dimension or two 20
21. • https://wettel.github.io/ codecity-download.html • Java, C++, C# • Limited to non- commercial use 21
22. • http://gource.io/ • Evolution of codebase • Works with multiple source code repositories 22
23. • Concept introduced by Ward Cunningham • Quick, Visual way to scan for patterns in code 23
24. package org.apache.tomcat.websocket; import javax.websocket.Decoder; public class DecoderEntry { private final Class<?> clazz; private final Class<? extends Decoder> decoderClazz; public DecoderEntry(Class<?> clazz, Class<? extends Decoder> decoderClazz) { this.clazz = clazz; this.decoderClazz = decoderClazz; } public Class<?> getClazz() { return clazz; } public Class<? extends Decoder> getDecoderClazz() { return decoderClazz; } }
25. package org.apache.tomcat.websocket; import java.nio.ByteBuffer; import javax.websocket.SendHandler; class MessagePart { private final boolean fin; private final int rsv; private final byte opCode; private final ByteBuffer payload; private final SendHandler intermediateHandler; private volatile SendHandler endHandler; private final long blockingWriteTimeoutExpiry; public MessagePart( boolean fin, int rsv, byte opCode, ByteBuffer payload, SendHandler intermediateHandler, SendHandler endHandler, long blockingWriteTimeoutExpiry) { this.fin = fin; this.rsv = rsv; this.opCode = opCode; this.payload = payload; this.intermediateHandler = intermediateHandler; this.endHandler = endHandler; this.blockingWriteTimeoutExpiry = blockingWriteTimeoutExpiry; } public boolean isFin() { return fin; } public int getRsv() { return rsv; } public byte getOpCode() { return opCode; } public ByteBuffer getPayload() { return payload; } public SendHandler getIntermediateHandler() { return intermediateHandler; } public SendHandler getEndr() { return endHandler; } public void setEndHandler(SendHandler endHandler) { this.endHandler = endHandler; } public long getBlockingWriteTimeoutExpiry() { return blockingWriteTimeoutExpiry; } }
26. DecoderEntry.class: ;;{;;(){;;}{;}(){;}} MessagePart.class: ;;;{;;;;;;;(){;;;;;;;}(){;}(){;}(){;}(){;}(){;}(){;}(){;}(){;}} MessageHandlerResult.class: ;;{;;(){;;}(){;}(){;}} WsIOException.class: ;;;{;;(){;}(){;}} WsPongMessage.class: ;;;{;(){;;;}(){;}}
27. NestedIf.class: ;;{(){{{{{{}}}}}}} LotsOfImports.class: ;;;;;;;;;;;;;;;{;;;;;;;(){;;;;;;;}(){;}} Constants.class: ;;{;;;;;;;;;;;;;;;;;;;;;;;;} ReallyBigMethod.class: ;;;{;;(){;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;}(){;}}
28. • My version is at http://github.com/ vanderfox/signaturesurvey • Attempting to make it language agnostic • Pass in list of characters to scan for 28
29. 29 • Found at: http://logstalgia.io/ • Visualize traffic to a web server
31. • Keep track of your changes and experiments • Analyze history of code to date • Branch to isolate changes 31
34. 34 • See what your upstream/ Downstream dependencies are • Specifically, see what you will break with changes
36. • Intercept requests between machines • Inspect Headers • Look at packets • record conversations between machines 36
39. • Cloudwatch • StackDriver Monitoring • Datadog • LogicMonitor 39
40. • Splunk • ELK • Graylog 40
41. • Poke the box • Unit Testing for code • End to End testing Tools for Infrastructure • Drive full stacks or UIs 41
42. • Validate the knowledge learned • Duplicate the artifact 42
43. • Ansible • Salt • Chef • Puppet • Capistrano 43
44. • Much better for automation than shell scripts • Idempotent • Discovery is a repetitive activity 44
45. • https://en.wikipedia.org/wiki/Software_archaeology • http://c2.com/doc/SignatureSurvey/ • https://wettel.github.io/codecity-download.html • https://en.wikipedia.org/wiki/Infrastructure_as_Code 45
46. • Lee Fox • Email: lee.h.fox@gmail.com • Twitter: @FoxInATX 46 http://bit.ly/DevOpsArcheology