$30 off During Our Annual Pro Sale. View Details »

DevOpsDays Cuba 2017: DevOps Archeology

DevOpsDays Cuba
October 25, 2017
310

DevOpsDays Cuba 2017: DevOps Archeology

Author: Lee Fox
Summary: Back in at the 2001 OOPSLA Conference, the term Software Archeology was first coined. It gave a set of tools and processes for understanding a poorly documented and poorly maintained software architecture. It didn’t just work for the undocumented codebases, but ones that are incredibly large and complex, even if good documentation exists. The idea was to follow the true archeology with investigative work to understand the mindsets of the people who did the original implementations. Nowadays, the problem of understanding software still exists, but has also grown by a quite with the larger scales of production environments that are often times distributed or found in the cloud. So, this session will help the DevOps engineer walking into a new shop with tools and techniques to understand, not only the code, but the entire production system as well. This will help provide faster ramp time, higher quality software implementations, and more stable production environments.

DevOpsDays Cuba

October 25, 2017
Tweet

More Decks by DevOpsDays Cuba

Transcript

  1. Lee Fox
    @FoxInATX
    1

    View Slide

  2. • Cloud Architect at Infor
    • Agile, DevOps, ChatOps evangelist
    • Involved with Agile Austin, Cloud Austin
    • Alexa Developer
    • Amateur Chef
    2

    View Slide

  3. • Java development background
    • AWS background
    3

    View Slide

  4. • DevOps Archeology
    • Dig Techniques
    4

    View Slide

  5. 5

    View Slide

  6. 6

    View Slide

  7. 7

    View Slide

  8. 8

    View Slide

  9. 9

    View Slide

  10. • First appeared at 2001 OOPSLA
    conference as a workshop
    • Gain perspective and understanding into
    application behavior
    • Understand what predecessors were
    thinking
    10

    View Slide

  11. • First appeared at Austin DevOps, October 2017
    • Applying Concept to Infrastructure
    • Infrastructure as code
    • Learn what predecessors were thinking
    • Understand how systems work together
    11

    View Slide

  12. • This is about the artifacts—the technology
    • Culture and processes forthcoming in
    DevOps Anthropology
    12

    View Slide

  13. • Scripting languages
    • Ongoing documentation
    • Signature Analysis and
    software visualization
    • Reverse Engineering tools
    • Operating System level
    traces
    • Coupling Analysis
    • Source Control
    • Search Engines
    • IDE file browsing
    • Unit testing
    • API documentation
    generation tools
    • Debuggers
    • Dependency Analysis
    13

    View Slide

  14. • Monitoring tools
    • Log Aggregation
    • Automation tools
    • Testing Tools

    View Slide

  15. 15

    View Slide

  16. 16

    View Slide

  17. • Start by observing the box
    • Describe it
    • Conduct experiments
    17

    View Slide

  18. • Isolate the artifact
    • Apply stimuli to the artifact
    • Measure the response of the
    artifact
    • Repeat as necessary
    18

    View Slide

  19. 19

    View Slide

  20. • Visualizations Provide easy ways
    to understand code
    • A picture is worth 1000 words
    • Visualizations are focused along a
    single dimension or two
    20

    View Slide

  21. • https://wettel.github.io/
    codecity-download.html
    • Java, C++, C#
    • Limited to non-
    commercial use
    21

    View Slide

  22. • http://gource.io/
    • Evolution of codebase
    • Works with multiple
    source code
    repositories
    22

    View Slide

  23. • Concept introduced by Ward
    Cunningham
    • Quick, Visual way to scan for
    patterns in code
    23

    View Slide

  24. package org.apache.tomcat.websocket;
    import javax.websocket.Decoder;
    public class DecoderEntry {
    private final Class> clazz;
    private final Class extends Decoder> decoderClazz;
    public DecoderEntry(Class> clazz, Class extends Decoder> decoderClazz) {
    this.clazz = clazz;
    this.decoderClazz = decoderClazz;
    }
    public Class> getClazz() {
    return clazz;
    }
    public Class extends Decoder> getDecoderClazz() {
    return decoderClazz;
    }
    }

    View Slide

  25. package org.apache.tomcat.websocket;
    import java.nio.ByteBuffer;
    import javax.websocket.SendHandler;
    class MessagePart {
    private final boolean fin;
    private final int rsv;
    private final byte opCode;
    private final ByteBuffer payload;
    private final SendHandler intermediateHandler;
    private volatile SendHandler endHandler;
    private final long blockingWriteTimeoutExpiry;
    public MessagePart( boolean fin, int rsv, byte opCode, ByteBuffer payload, SendHandler intermediateHandler,
    SendHandler endHandler, long blockingWriteTimeoutExpiry) {
    this.fin = fin;
    this.rsv = rsv;
    this.opCode = opCode;
    this.payload = payload;
    this.intermediateHandler = intermediateHandler;
    this.endHandler = endHandler;
    this.blockingWriteTimeoutExpiry = blockingWriteTimeoutExpiry;
    }
    public boolean isFin() { return fin; }
    public int getRsv() { return rsv; }
    public byte getOpCode() { return opCode; }
    public ByteBuffer getPayload() { return payload; }
    public SendHandler getIntermediateHandler() { return intermediateHandler; }
    public SendHandler getEndr() { return endHandler; }
    public void setEndHandler(SendHandler endHandler) { this.endHandler = endHandler; }
    public long getBlockingWriteTimeoutExpiry() { return blockingWriteTimeoutExpiry; }
    }

    View Slide

  26. DecoderEntry.class: ;;{;;(){;;}{;}(){;}}
    MessagePart.class: ;;;{;;;;;;;(){;;;;;;;}(){;}(){;}(){;}(){;}(){;}(){;}(){;}(){;}}
    MessageHandlerResult.class: ;;{;;(){;;}(){;}(){;}}
    WsIOException.class: ;;;{;;(){;}(){;}}
    WsPongMessage.class: ;;;{;(){;;;}(){;}}

    View Slide

  27. NestedIf.class: ;;{(){{{{{{}}}}}}}
    LotsOfImports.class: ;;;;;;;;;;;;;;;{;;;;;;;(){;;;;;;;}(){;}}
    Constants.class: ;;{;;;;;;;;;;;;;;;;;;;;;;;;}
    ReallyBigMethod.class: ;;;{;;(){;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;}(){;}}

    View Slide

  28. • My version is at http://github.com/
    vanderfox/signaturesurvey
    • Attempting to make it language
    agnostic
    • Pass in list of characters to scan for
    28

    View Slide

  29. 29
    • Found at: http://logstalgia.io/
    • Visualize traffic to a web server

    View Slide

  30. 30

    View Slide

  31. • Keep track of your changes and
    experiments
    • Analyze history of code to date
    • Branch to isolate changes
    31

    View Slide

  32. 32

    View Slide

  33. 33

    View Slide

  34. 34
    • See what your upstream/
    Downstream dependencies are
    • Specifically, see what you will
    break with changes

    View Slide

  35. 35

    View Slide

  36. • Intercept requests between
    machines
    • Inspect Headers
    • Look at packets
    • record conversations between
    machines
    36

    View Slide

  37. 37

    View Slide

  38. 38

    View Slide

  39. • Cloudwatch
    • StackDriver Monitoring
    • Datadog
    • LogicMonitor
    39

    View Slide

  40. • Splunk
    • ELK
    • Graylog
    40

    View Slide

  41. • Poke the box
    • Unit Testing for code
    • End to End testing Tools for
    Infrastructure
    • Drive full stacks or UIs
    41

    View Slide

  42. • Validate the knowledge learned
    • Duplicate the artifact
    42

    View Slide

  43. • Ansible
    • Salt
    • Chef
    • Puppet
    • Capistrano
    43

    View Slide

  44. • Much better for automation than shell
    scripts
    • Idempotent
    • Discovery is a repetitive activity
    44

    View Slide

  45. • https://en.wikipedia.org/wiki/Software_archaeology
    • http://c2.com/doc/SignatureSurvey/
    • https://wettel.github.io/codecity-download.html
    • https://en.wikipedia.org/wiki/Infrastructure_as_Code
    45

    View Slide

  46. • Lee Fox
    • Email: [email protected]
    • Twitter: @FoxInATX
    46
    http://bit.ly/DevOpsArcheology

    View Slide