Upgrade to Pro — share decks privately, control downloads, hide ads and more …

The Open Source Developer's Toolbox

Tyler L
February 05, 2017

The Open Source Developer's Toolbox

Tyler L

February 05, 2017
Tweet

More Decks by Tyler L

Other Decks in Technology

Transcript

  1. The Open Source
    Developer’s Toolbox
    Open Source 101, February 2017
    Tyler Langlois
    Infrastructure Engineer, Elastic

    View full-size slide

  2. $ whois tylerjl
    ● Infrastructure engineering/devops
    at Elastic
    ○ Operations and software dev
    ● Other pursuits
    ○ Overwatch Zenyatta main,
    has a Mork-chi named Morty
    normal
    person

    View full-size slide

  3. Who is This For?
    ● What OSS tools are out
    there?
    ● What are they good
    for?
    ● Should I pick x or y?
    ● What are people
    using?

    View full-size slide

  4. Remember!
    The goal is usability!

    View full-size slide

  5. Background
    ~ 400
    Elasticsearch,
    logstash
    Kibana
    Beats
    Logstash
    Infra/Devops

    View full-size slide

  6. Background
    Elasticsearch,
    logstash
    Kibana
    Beats
    Logstash
    Infra/Devops
    ● CI
    ● Dependencies
    ● Toolchains
    ● Distribution

    View full-size slide

  7. Languages
    Stricter, more
    controlled
    Dynamic, diverse
    libraries, variety
    of use cases
    Specialized,
    well-suited for
    particular uses

    View full-size slide

  8. Code and documentation
    ● Remember, optimize for your productivity
    ● Bear in mind some languages (Java?) will dictate an effective
    environment (i.e., IntelliJ debugger)
    ● OS isn’t a hard prerequisite for many authoring options
    ● Even OS-specific dev technologies (see Docker) are going
    cross-platform (ish)
    Writing

    View full-size slide

  9. Writing
    CLI
    ● +
    ○ Lightweight
    ○ Composable
    ● -
    ○ Features like refactoring
    ○ Learning curve
    IDE
    ● +
    ○ Featureful
    ○ Tight language integration
    ● -
    ○ Complexity
    ○ Obfuscation
    To IDE or not IDE?

    View full-size slide

  10. Writing
    CLI IDE
    To IDE or not IDE?
    Eclipse
    Atom
    Language-specific (pycharm,
    rubymine, etc.)
    Vim
    Emacs
    nano… I guess?

    View full-size slide

  11. Writing
    Vim
    ● Verb-like editing
    ● Standard
    ● Revitalized with Neovim
    ● Diverse range of language
    plugins
    ● Arcane for beginners
    ● Feat. like refactoring = poor

    View full-size slide

  12. Writing
    Emacs
    ● Strong, pluggable codebase
    ● Hotkey-driven
    ● Equally diverse plugin
    community as Vim
    ● Added functionality (IRC, etc.)
    ● Arcane for beginners
    ● Feat. like refactoring = poor

    View full-size slide

  13. Writing
    Eclipse
    ● Tight language integration
    ● Pluggable
    ● Integrates SCM
    ● Strong support for langs like
    Java
    ● Can become resource hog
    ● Hides details under the covers

    View full-size slide

  14. Writing
    Atom
    ● Latest addition, .js based
    ● Somewhere between CLI and
    IDE load
    ● Sponsored + Developed at
    Github
    ● +/- Javascript

    View full-size slide

  15. Writing
    Traditional Tools
    ● SVN, CVS
    ○ Remote SCM server
    ○ Distributed development
    happens via locking
    ○ Only relevant pieces
    checked out
    Modern Replacements
    ● Git, Mercurial
    ○ Server + local components
    separate
    ○ Lightweight branching
    ○ Distributed development
    ○ De facto OSS winner
    Source Control

    View full-size slide

  16. Writing
    Source Control

    View full-size slide

  17. Writing
    Git, Mercurial
    ● Immutable and graph-like history
    ● Interface options
    ○ IDE, CLI, dedicated software
    ● Git vs. Mercurial?
    ○ git: low-level swiss army
    knife
    ○ hg: more user-friendly, less
    adoption

    View full-size slide

  18. Writing
    Git, Mercurial
    ● Native tools aren’t bad
    ● CLI / IDEs create identical histories;
    users can collaborate with either
    ● It’s worthwhile to brush up on the
    data model to avoid major blowups
    (Google “the git parable”)

    View full-size slide

  19. Language + Editor + SCM + … ?
    ● Additional command-line tools?
    ○ Homebrew, chocolatey, $package_manager
    ● Shell proficiency
    ○ Bash is standard, but beefier options are available
    ○ Zsh, fish
    Utilities

    View full-size slide

  20. Language + Editor + SCM + … ?
    ● Language version manager
    ○ Versions differ significantly, many projects require a specific
    major version
    Utilities
    jenv, sdkman
    nvm
    Rvm,
    rbenv
    virtualenv
    stack
    … ?

    View full-size slide

  21. Services
    Why?
    Where do you:
    ● Test the software?
    ● Build the software?
    ● Run the software?
    ● Document the software?
    ● Collaborate on the software?
    The sad reality: there is no free lunch

    View full-size slide

  22. Services
    A Word of Warning:

    View full-size slide

  23. Services
    Collaboration - Cloud
    GitHub
    ● De facto OSS repository
    ● Git (no mercurial)
    ● Model is pricing for private
    repos
    Bitbucket
    ● Smaller but also popular
    ● git + mercurial

    View full-size slide

  24. Services
    Collaboration - Self-Hosted
    GitLab
    ● Self-hosted GitHub
    alternative
    gogs
    ● Self-contained go service
    gitolite
    ● CLI-based service

    View full-size slide

  25. Services
    Testing
    $ rspec # ruby
    $ py.test # python
    $ go test # go
    ...many more

    View full-size slide

  26. Services
    Testing
    Travis CI (Cloud)
    ● Free for OSS
    ● Tightly integrated into
    GitHub
    Jenkins (Self-hosted)
    ● Most common self-hosted
    option

    View full-size slide

  27. Services
    Testing

    View full-size slide

  28. Databases
    Services

    View full-size slide

  29. Databases
    Services
    1. Don’t try and solve Google-sized scaling issues first
    2. SQL isn’t uncool
    3. It’s all about the shape of your data
    4. Know your access patterns
    5. SAFETY
    6. ???
    7. Profit

    View full-size slide

  30. Databases
    Services
    ● SQL
    ○ Postgresql, MySQL
    ○ Not hip, but been around a long time
    ○ Why are they still relevant?
    ■ Lots of data is relational!
    ○ SQLite convenient option for local development
    ○ Well-documented for self-hosting or hosted (RDS)
    solutions

    View full-size slide

  31. Databases
    Services
    ● Search
    ○ Lucene is the grandaddy
    ○ Solr, Elasticsearch
    ○ RESTful, JSON-oriented, distributed
    ○ Useful to make an application:
    ■ Autocomplete
    ■ Highlight
    ■ Real fast

    View full-size slide

  32. Databases
    Services
    ● Timeseries
    ○ For when you’ve just got numbers
    ○ Tuned to numerics

    View full-size slide

  33. Databases
    Services
    ● Other NoSQL
    ○ MongoDB
    ■ Easy to get started, scalable
    ○ Hadoop Ecosystem
    ■ For really, REALLY big data
    ■ HDFS, MapReduce, etc.
    ○ + many more

    View full-size slide

  34. This is just the surface

    View full-size slide

  35. Thank You!
    ● GitHub / Speakerdeck
    ○ tylerjl
    ○ (vim configs at
    tylerjl/dotfiles on
    github)
    ● IRC / Twitter
    ○ leothrix
    ● tjll.net

    View full-size slide