The Well Rounded Engineer

What does it take to do a software engineering job well? Are you interested in deepening your skills but haven’t had the opportunity to lay a solid foundation? It turns out the high-level answer to this is simple, and it is a skill and knowledge-based system balanced on these two load-bearing pillars:

1. Individual technical ability
2. Collaborating and working with others

Take either one away, and it comes crashing down. In this talk, we aim to deconstruct the former: individual technical skill. What is the foundation on which lays the pyramid of skills?

We will take a look at a roster of abilities, and we will talk about one or two concrete steps you can take to make progress in each.

This roster is aimed primarily at software engineers who want to write modern web-based systems.

Transcript

1. The Well Rounded
   Engineer: Critical Path
   One step at a time. !
   

   View Slide

2. Swanand Pagnis
   " Director of Engineering at Deserve
   meetup.com/Bangalore-Ruby-Users-Group/
   info.pagnis.in
   % postgres-workshop.com
   

   View Slide

3. View Slide

4. This talk is about
   crossing or getting near
   the horizontal blue line.
   

   View Slide

5. Target Audience:
   Web based application developers,
   with a slight lean towards back-end developers.
   

   View Slide

6. A core skill or
   category of core skills
   What can you do to
   improve them?
   

   View Slide

7. Some interview skills
   What can you do to
   improve them?
   

   View Slide

8. Before we begin…
   

   View Slide

9. Some process tips:
   

   View Slide

10. Find peers; learn
    together
    Be accountable
    Hold them accountable
    

    View Slide

11. Get rid of the trial-
    and-error method
    Take a very analytical
    approach
    

    View Slide

12. Learn how to read
    research papers, specially
    older papers.
    Bleeding edge is often
    overrated
    

    View Slide

13. Write summaries of
    things you study
    Spaced repetition helps
    you get over the I'll-
    never-use-this-syndrome
    

    View Slide

14. 1. Paradigm Polyglot
    

    View Slide

15. Be proficient in two
    paradigms.
    

    View Slide

16. Preferably OOP and FP.
    

    View Slide

17. Proficient =
    Being able to write non-
    trivial programs.
    

    View Slide

18. e.g. A Message Broker,
    A Web Crawler, An API
    Wrapper etc.
    

    View Slide

19. Boundaries are being
    blurred by languages
    like Go and Scala.
    

    View Slide

20. - Go, Java, Javascript
    - Ruby, Python
    - Haskell, OCaml
    

    View Slide

21. Pick one from each.
    Choose based on
    comfort & current skill.
    

    View Slide

22. Bonus:
    - Prolog, Factor, Idris
    - Lisp, Scheme, Clojure
    

    View Slide

23. My personal choice:
    Ruby, OCaml, Prolog
    

    View Slide

24. For Ruby:
    Avdi Grimm's
    Confident Ruby
    

    View Slide

25. For OCaml:
    Real World OCaml by
    Yaron Minsky et al
    

    View Slide

26. For Haskell:
    Haskell Book by
    Chris A. and Julie M.
    

    View Slide

27. For Prolog:
    Exercism Track and 99
    Prolog Problems
    

    View Slide

28. Strongly
    recommended:
    Prof. Dan Grossman's "Programming Languages"
    on Coursera. Life changing course.
    

    View Slide

29. This course will
    help you understand
    programming languages
    on a fundamental level.
    

    View Slide

30. 2. Database Polyglot
    

    View Slide

31. Be proficient in using
    two types of databases.
    

    View Slide

32. The two types are:
    RDBMS and NoSQL*.
    Duh.
    *NoSQL is too broad, but again, pick something closer to your
    preferred stack.
    

    View Slide

33. Conversant =
    - Model your data
    - Be performance aware
    - Write raw SQL
    

    View Slide

34. Proficient =
    - Idiomatic design
    - Be operations aware
    - Grok SQL
    

    View Slide

35. In general, proficiency is
    about understanding
    trade-offs.
    

    View Slide

36. In depth understanding
    of SQL is a competitive
    advantage.
    

    View Slide

37. - MySQL, PostgreSQL
    - Dynamo, Bigtable
    - Redis, Spanner, Neo4j
    

    View Slide

38. For PostgreSQL:
    Dmitri Fontaine's
    Mastering PostgreSQL in
    Application Development
    

    View Slide

39. For MySQL:
    Baron Schwartz's High
    Performance MySQL
    

    View Slide

40. For DynamoDB:
    DynamoDBGuide.com,
    AWS's Dynamo Paper
    

    View Slide

41. Books and guides are
    just a start, specially
    when it comes to DBs.
    

    View Slide

42. 3. Protocol Polyglot
    

    View Slide

43. Be proficient in using
    two types of network
    protocols. )
    

    View Slide

44. Preferably
    TCP/IP and HTTP.
    

    View Slide

45. Proficient =
    - How it works
    - How to debug issues
    - Design considerations
    

    View Slide

46. Proficient =
    - Familiarity w/ SSL, TLS
    - App layer integration
    - Sockets & ports
    

    View Slide

47. Start here:
    Ilya Grigorik's High
    Performance Browser
    Networking
    

    View Slide

48. For TCP and Sockets:
    Jesse Storimer's
    Working With TCP
    Sockets
    

    View Slide

49. For HTTP:
    - Read RFC 7231
    - Optionally: RFC 2616
    

    View Slide

50. 4. Proficiency with
    build tooling,
    packaging and
    distribution of software
    

    View Slide

51. How does the code you
    write reach your users?
    

    View Slide

52. Proficiency =
    Being able to use these
    tools and tweak,
    change what's needed
    

    View Slide

53. For a Go programmer:
    Intimately understand
    go build
    

    View Slide

54. For a Rubyist:
    Understand how Gems
    are packaged, distributed,
    and loaded at runtime.
    

    View Slide

55. For a Rubyist:
    guides.rubygems.org
    

    View Slide

56. For everyone:
    Write a simple build
    tool.
    

    View Slide

57. What does it do?
    - Takes an opinionated
    directory structure
    - Compiles / runs your
    multi-file program
    

    View Slide

58. What does it do next?
    - Tackle simple
    dependencies
    - Build a jar / zip like
    package
    

    View Slide

59. 5. Ability to analyse,
    reason about and
    debug programs.
    

    View Slide

60. Reading & reviewing code
    Debug running programs
    Observability &
    instrumentation
    

    View Slide

61. Reading & reviewing
    code
    

    View Slide

62. Develop a habit of reading,
    reviewing, and explaining
    every single line of code
    you write or review.
    

    View Slide

63. This includes
    whitespaces, empty
    lines, config files,
    dotfiles, comments
    

    View Slide

64. The 5-whys are your
    friend.
    

    View Slide

65. Develop a habit of
    reading library and open
    source code. They're
    often idiomatic.
    

    View Slide

66. Debug running
    programs
    

    View Slide

67. Use GDB or your
    favourite step-by-step
    debugger on a process
    running in QA / Staging
    

    View Slide

68. Setup IntelliJ or your
    favourite editor to
    trivially run debug mode,
    and step through code
    

    View Slide

69. Write a *very simple*
    sampling profiler for
    your language.
    

    View Slide

70. Julia Evans' talk about
    her building rbspy is full
    of good information
    

    View Slide

71. Optionally:
    Read the ptrace
    manpages. 3 times.
    

    View Slide

72. Observability and
    instrumentation
    

    View Slide

73. Inculcate observability
    in your process of
    writing software.
    

    View Slide

74. honeycomb.io 's
    Observability Guide.
    

    View Slide

75. Cindy Sridharan's
    Observability Book
    

    View Slide

76. How can I reliably
    answer a question
    about a system, without
    prying open the system.
    

    View Slide

77. 6. Deployments,
    Infrastructure, and
    DevOps
    

    View Slide

78. Proficiency !=
    Writing an Ansible clone
    

    View Slide

79. Proficiency =
    Understanding when
    you need Ansible clone
    

    View Slide

80. Write a simple
    Capistrano like
    deployment system
    

    View Slide

81. Keep it opinionated,
    and bare minimum.
    

    View Slide

82. 7. Familiarity with
    software architecture
    principles, and basic
    scaling techniques.
    

    View Slide

83. Understand how to
    design commonly used
    systems
    

    View Slide

84. e.g. How to use
    workers, queues, and
    async processing
    

    View Slide

85. e.g. How to design a
    data pipeline
    

    View Slide

86. e.g. How to design
    simple load balancers
    

    View Slide

87. e.g. MapReduce™ and
    similar data processing
    patterns
    

    View Slide

88. Martin Kleppmann's
    Designing Data
    Intensive Applications
    

    View Slide

89. Michael Keeling's
    Design It!
    

    View Slide

90. Solve a few "scaling"
    problems from Gayle
    McDowell's Cracking
    the Coding Interview.
    

    View Slide

91. 8. Ability to write toy
    compilers,
    interpreters, and
    parsers
    

    View Slide

92. View Slide

93. This is the most fun
    section.
    

    View Slide

94. This is also a very useful
    interview skill.
    

    View Slide

95. Start with parsers, then
    interpreters, and finally
    compilers.
    

    View Slide

96. Do not start at book
    

    View Slide

97. Work through Graham
    Hutton's "Higher Order
    Functions for Parsing"
    

    View Slide

98. Then work through
    Peter Norvig's Lispy:
    norvig.com/lispy.html
    

    View Slide

99. Once you're confident,
    attempt to write a
    JSON or YAML parser.
    

    View Slide

100. Recursive descent
     parsers open you up to a
     whole new coding style
     

     View Slide

101. Racket/Scheme/
     Haskell/OCaml are ideal
     for writing compilers.
     

     View Slide

102. For compilers, work
     through "An
     incremental approach to
     compiler construction"
     

     View Slide

103. 9. Ability to write toy
     games
     

     View Slide

104. Again a very useful
     interview skill
     

     View Slide

105. Games can have
     arbitrary changing
     requirements.
     

     View Slide

106. Games typically adhere
     to a specific set of rules
     

     View Slide

107. With games, come
     game playing bots.
     

     View Slide

108. Write a simple game with
     minimal or ascii graphics:
     Snake, Minesweeper,
     Jumper Frog, etc
     

     View Slide

109. Avoid unity or other
     frameworks, try from
     first principles.
     

     View Slide

110. Then write a bot that
     plays that game
     

     View Slide

111. If you like doing that,
     read about general
     game play: Bots that
     can play *any* game.
     

     View Slide

112. 10. Ability to
     understand algorithmic
     analysis and solve
     algorithmic problems
     

     View Slide

113. Yet another useful
     interview skill
     

     View Slide

114. But an equally useful
     programming skill
     

     View Slide

115. Understand the
     fundamental approach
     of solving problems
     recursively.
     

     View Slide

116. Understand the
     "master method" to
     analyse recursive
     solutions.
     

     View Slide

117. Cracking The Coding
     Interview
     

     View Slide

118. InterviewCake
     

     View Slide

119. Tim Roughgarden's
     Course on Coursera
     - This is a challenging
     course
     

     View Slide

120. Putting it all
     together
     

     View Slide

121. Make a plan
     One thing at a time
     Focus hard on it
     Prioritize fundamental,
     foundational skills
     

     View Slide

122. Do not go too far away
     from current skillset
     Get in the habit of writing
     things down: notes,
     reviews, retrospectives.
     

     View Slide

123. Process tips again,
     they're that important
     

     View Slide

124. Find peers; learn
     together
     Be accountable
     Hold them accountable
     

     View Slide

125. Get rid of the trial-
     and-error method
     Take a very analytical
     approach
     

     View Slide

126. Learn how to read
     research papers, specially
     older papers.
     Bleeding edge is often
     overrated
     

     View Slide

127. Write summaries of
     things you study
     Spaced repetition will get
     you get over the I'll-
     never-use-this syndrome
     

     View Slide

128. Thank you!
     Tell me about your process.
     

     View Slide

129. Swanand Pagnis
     " Director of Engineering at Deserve
     meetup.com/Bangalore-Ruby-Users-Group/
     info.pagnis.in
     % postgres-workshop.com
     

     View Slide