Upgrade to Pro — share decks privately, control downloads, hide ads and more …

The Open Source Developer's Toolbox

Tyler L
February 05, 2017

The Open Source Developer's Toolbox

Tyler L

February 05, 2017
Tweet

More Decks by Tyler L

Other Decks in Technology

Transcript

  1. The Open Source
    Developer’s Toolbox
    Open Source 101, February 2017
    Tyler Langlois
    Infrastructure Engineer, Elastic

    View Slide

  2. $ whois tylerjl
    ● Infrastructure engineering/devops
    at Elastic
    ○ Operations and software dev
    ● Other pursuits
    ○ Overwatch Zenyatta main,
    has a Mork-chi named Morty
    normal
    person

    View Slide

  3. Who is This For?
    ● What OSS tools are out
    there?
    ● What are they good
    for?
    ● Should I pick x or y?
    ● What are people
    using?

    View Slide

  4. Remember!
    The goal is usability!

    View Slide

  5. Background
    ~ 400
    Elasticsearch,
    logstash
    Kibana
    Beats
    Logstash
    Infra/Devops

    View Slide

  6. Background
    Elasticsearch,
    logstash
    Kibana
    Beats
    Logstash
    Infra/Devops
    ● CI
    ● Dependencies
    ● Toolchains
    ● Distribution

    View Slide

  7. Languages

    View Slide

  8. Languages
    Stricter, more
    controlled
    Dynamic, diverse
    libraries, variety
    of use cases
    Specialized,
    well-suited for
    particular uses

    View Slide

  9. Writing

    View Slide

  10. Code and documentation
    ● Remember, optimize for your productivity
    ● Bear in mind some languages (Java?) will dictate an effective
    environment (i.e., IntelliJ debugger)
    ● OS isn’t a hard prerequisite for many authoring options
    ● Even OS-specific dev technologies (see Docker) are going
    cross-platform (ish)
    Writing

    View Slide

  11. Writing
    CLI
    ● +
    ○ Lightweight
    ○ Composable
    ● -
    ○ Features like refactoring
    ○ Learning curve
    IDE
    ● +
    ○ Featureful
    ○ Tight language integration
    ● -
    ○ Complexity
    ○ Obfuscation
    To IDE or not IDE?

    View Slide

  12. Writing
    CLI IDE
    To IDE or not IDE?
    Eclipse
    Atom
    Language-specific (pycharm,
    rubymine, etc.)
    Vim
    Emacs
    nano… I guess?

    View Slide

  13. Writing
    Vim
    ● Verb-like editing
    ● Standard
    ● Revitalized with Neovim
    ● Diverse range of language
    plugins
    ● Arcane for beginners
    ● Feat. like refactoring = poor

    View Slide

  14. Writing
    Emacs
    ● Strong, pluggable codebase
    ● Hotkey-driven
    ● Equally diverse plugin
    community as Vim
    ● Added functionality (IRC, etc.)
    ● Arcane for beginners
    ● Feat. like refactoring = poor

    View Slide

  15. Writing
    Eclipse
    ● Tight language integration
    ● Pluggable
    ● Integrates SCM
    ● Strong support for langs like
    Java
    ● Can become resource hog
    ● Hides details under the covers

    View Slide

  16. Writing
    Atom
    ● Latest addition, .js based
    ● Somewhere between CLI and
    IDE load
    ● Sponsored + Developed at
    Github
    ● +/- Javascript

    View Slide

  17. Writing
    Traditional Tools
    ● SVN, CVS
    ○ Remote SCM server
    ○ Distributed development
    happens via locking
    ○ Only relevant pieces
    checked out
    Modern Replacements
    ● Git, Mercurial
    ○ Server + local components
    separate
    ○ Lightweight branching
    ○ Distributed development
    ○ De facto OSS winner
    Source Control

    View Slide

  18. Writing
    Source Control

    View Slide

  19. Writing
    Git, Mercurial
    ● Immutable and graph-like history
    ● Interface options
    ○ IDE, CLI, dedicated software
    ● Git vs. Mercurial?
    ○ git: low-level swiss army
    knife
    ○ hg: more user-friendly, less
    adoption

    View Slide

  20. Writing
    Git, Mercurial
    ● Native tools aren’t bad
    ● CLI / IDEs create identical histories;
    users can collaborate with either
    ● It’s worthwhile to brush up on the
    data model to avoid major blowups
    (Google “the git parable”)

    View Slide

  21. Utilities

    View Slide

  22. Language + Editor + SCM + … ?
    ● Additional command-line tools?
    ○ Homebrew, chocolatey, $package_manager
    ● Shell proficiency
    ○ Bash is standard, but beefier options are available
    ○ Zsh, fish
    Utilities

    View Slide

  23. Language + Editor + SCM + … ?
    ● Language version manager
    ○ Versions differ significantly, many projects require a specific
    major version
    Utilities
    jenv, sdkman
    nvm
    Rvm,
    rbenv
    virtualenv
    stack
    … ?

    View Slide

  24. Services

    View Slide

  25. Services
    Why?
    Where do you:
    ● Test the software?
    ● Build the software?
    ● Run the software?
    ● Document the software?
    ● Collaborate on the software?
    The sad reality: there is no free lunch

    View Slide

  26. Services
    A Word of Warning:

    View Slide

  27. Services
    Collaboration - Cloud
    GitHub
    ● De facto OSS repository
    ● Git (no mercurial)
    ● Model is pricing for private
    repos
    Bitbucket
    ● Smaller but also popular
    ● git + mercurial

    View Slide

  28. Services
    Collaboration - Self-Hosted
    GitLab
    ● Self-hosted GitHub
    alternative
    gogs
    ● Self-contained go service
    gitolite
    ● CLI-based service

    View Slide

  29. Services
    Testing
    $ rspec # ruby
    $ py.test # python
    $ go test # go
    ...many more

    View Slide

  30. Services
    Testing
    Travis CI (Cloud)
    ● Free for OSS
    ● Tightly integrated into
    GitHub
    Jenkins (Self-hosted)
    ● Most common self-hosted
    option

    View Slide

  31. Services
    Testing

    View Slide

  32. ?

    View Slide

  33. Databases
    Services

    View Slide

  34. Databases
    Services
    1. Don’t try and solve Google-sized scaling issues first
    2. SQL isn’t uncool
    3. It’s all about the shape of your data
    4. Know your access patterns
    5. SAFETY
    6. ???
    7. Profit

    View Slide

  35. Databases
    Services
    ● SQL
    ○ Postgresql, MySQL
    ○ Not hip, but been around a long time
    ○ Why are they still relevant?
    ■ Lots of data is relational!
    ○ SQLite convenient option for local development
    ○ Well-documented for self-hosting or hosted (RDS)
    solutions

    View Slide

  36. Databases
    Services
    ● Search
    ○ Lucene is the grandaddy
    ○ Solr, Elasticsearch
    ○ RESTful, JSON-oriented, distributed
    ○ Useful to make an application:
    ■ Autocomplete
    ■ Highlight
    ■ Real fast

    View Slide

  37. Databases
    Services
    ● Timeseries
    ○ For when you’ve just got numbers
    ○ Tuned to numerics

    View Slide

  38. Databases
    Services
    ● Other NoSQL
    ○ MongoDB
    ■ Easy to get started, scalable
    ○ Hadoop Ecosystem
    ■ For really, REALLY big data
    ■ HDFS, MapReduce, etc.
    ○ + many more

    View Slide

  39. This is just the surface

    View Slide

  40. Questions?

    View Slide

  41. Thank You!
    ● GitHub / Speakerdeck
    ○ tylerjl
    ○ (vim configs at
    tylerjl/dotfiles on
    github)
    ● IRC / Twitter
    ○ leothrix
    ● tjll.net

    View Slide