The Open Source Developer's Toolbox

F43919144cdcddd7ba50e46f71667d08?s=47 Tyler L
February 05, 2017

The Open Source Developer's Toolbox

F43919144cdcddd7ba50e46f71667d08?s=128

Tyler L

February 05, 2017
Tweet

Transcript

  1. The Open Source Developer’s Toolbox Open Source 101, February 2017

    Tyler Langlois Infrastructure Engineer, Elastic
  2. $ whois tylerjl • Infrastructure engineering/devops at Elastic ◦ Operations

    and software dev • Other pursuits ◦ Overwatch Zenyatta main, has a Mork-chi named Morty normal person
  3. Who is This For? • What OSS tools are out

    there? • What are they good for? • Should I pick x or y? • What are people using?
  4. Remember! The goal is usability!

  5. Background ~ 400 Elasticsearch, logstash Kibana Beats Logstash Infra/Devops

  6. Background Elasticsearch, logstash Kibana Beats Logstash Infra/Devops • CI •

    Dependencies • Toolchains • Distribution
  7. Languages

  8. Languages Stricter, more controlled Dynamic, diverse libraries, variety of use

    cases Specialized, well-suited for particular uses
  9. Writing

  10. Code and documentation • Remember, optimize for your productivity •

    Bear in mind some languages (Java?) will dictate an effective environment (i.e., IntelliJ debugger) • OS isn’t a hard prerequisite for many authoring options • Even OS-specific dev technologies (see Docker) are going cross-platform (ish) Writing
  11. Writing CLI • + ◦ Lightweight ◦ Composable • -

    ◦ Features like refactoring ◦ Learning curve IDE • + ◦ Featureful ◦ Tight language integration • - ◦ Complexity ◦ Obfuscation To IDE or not IDE?
  12. Writing CLI IDE To IDE or not IDE? Eclipse Atom

    Language-specific (pycharm, rubymine, etc.) Vim Emacs nano… I guess?
  13. Writing Vim • Verb-like editing • Standard • Revitalized with

    Neovim • Diverse range of language plugins • Arcane for beginners • Feat. like refactoring = poor
  14. Writing Emacs • Strong, pluggable codebase • Hotkey-driven • Equally

    diverse plugin community as Vim • Added functionality (IRC, etc.) • Arcane for beginners • Feat. like refactoring = poor
  15. Writing Eclipse • Tight language integration • Pluggable • Integrates

    SCM • Strong support for langs like Java • Can become resource hog • Hides details under the covers
  16. Writing Atom • Latest addition, .js based • Somewhere between

    CLI and IDE load • Sponsored + Developed at Github • +/- Javascript
  17. Writing Traditional Tools • SVN, CVS ◦ Remote SCM server

    ◦ Distributed development happens via locking ◦ Only relevant pieces checked out Modern Replacements • Git, Mercurial ◦ Server + local components separate ◦ Lightweight branching ◦ Distributed development ◦ De facto OSS winner Source Control
  18. Writing Source Control

  19. Writing Git, Mercurial • Immutable and graph-like history • Interface

    options ◦ IDE, CLI, dedicated software • Git vs. Mercurial? ◦ git: low-level swiss army knife ◦ hg: more user-friendly, less adoption
  20. Writing Git, Mercurial • Native tools aren’t bad • CLI

    / IDEs create identical histories; users can collaborate with either • It’s worthwhile to brush up on the data model to avoid major blowups (Google “the git parable”)
  21. Utilities

  22. Language + Editor + SCM + … ? • Additional

    command-line tools? ◦ Homebrew, chocolatey, $package_manager • Shell proficiency ◦ Bash is standard, but beefier options are available ◦ Zsh, fish Utilities
  23. Language + Editor + SCM + … ? • Language

    version manager ◦ Versions differ significantly, many projects require a specific major version Utilities jenv, sdkman nvm Rvm, rbenv virtualenv stack … ?
  24. Services

  25. Services Why? Where do you: • Test the software? •

    Build the software? • Run the software? • Document the software? • Collaborate on the software? The sad reality: there is no free lunch
  26. Services A Word of Warning:

  27. Services Collaboration - Cloud GitHub • De facto OSS repository

    • Git (no mercurial) • Model is pricing for private repos Bitbucket • Smaller but also popular • git + mercurial
  28. Services Collaboration - Self-Hosted GitLab • Self-hosted GitHub alternative gogs

    • Self-contained go service gitolite • CLI-based service
  29. Services Testing $ rspec # ruby $ py.test # python

    $ go test # go ...many more
  30. Services Testing Travis CI (Cloud) • Free for OSS •

    Tightly integrated into GitHub Jenkins (Self-hosted) • Most common self-hosted option
  31. Services Testing

  32. ?

  33. Databases Services

  34. Databases Services 1. Don’t try and solve Google-sized scaling issues

    first 2. SQL isn’t uncool 3. It’s all about the shape of your data 4. Know your access patterns 5. SAFETY 6. ??? 7. Profit
  35. Databases Services • SQL ◦ Postgresql, MySQL ◦ Not hip,

    but been around a long time ◦ Why are they still relevant? ▪ Lots of data is relational! ◦ SQLite convenient option for local development ◦ Well-documented for self-hosting or hosted (RDS) solutions
  36. Databases Services • Search ◦ Lucene is the grandaddy ◦

    Solr, Elasticsearch ◦ RESTful, JSON-oriented, distributed ◦ Useful to make an application: ▪ Autocomplete ▪ Highlight ▪ Real fast
  37. Databases Services • Timeseries ◦ For when you’ve just got

    numbers ◦ Tuned to numerics
  38. Databases Services • Other NoSQL ◦ MongoDB ▪ Easy to

    get started, scalable ◦ Hadoop Ecosystem ▪ For really, REALLY big data ▪ HDFS, MapReduce, etc. ◦ + many more
  39. This is just the surface

  40. Questions?

  41. Thank You! • GitHub / Speakerdeck ◦ tylerjl ◦ (vim

    configs at tylerjl/dotfiles on github) • IRC / Twitter ◦ leothrix • tjll.net