The Open Source Developer's Toolbox

Transcript

1. The Open Source Developer’s Toolbox Open Source 101, February 2017 Tyler Langlois Infrastructure Engineer, Elastic
2. $ whois tylerjl • Infrastructure engineering/devops at Elastic ◦ Operations and software dev • Other pursuits ◦ Overwatch Zenyatta main, has a Mork-chi named Morty normal person
3. Who is This For? • What OSS tools are out there? • What are they good for? • Should I pick x or y? • What are people using?
4. Remember! The goal is usability!
5. Background ~ 400 Elasticsearch, logstash Kibana Beats Logstash Infra/Devops
6. Background Elasticsearch, logstash Kibana Beats Logstash Infra/Devops • CI • Dependencies • Toolchains • Distribution
7. Languages
8. Languages Stricter, more controlled Dynamic, diverse libraries, variety of use cases Specialized, well-suited for particular uses
9. Writing
10. Code and documentation • Remember, optimize for your productivity • Bear in mind some languages (Java?) will dictate an effective environment (i.e., IntelliJ debugger) • OS isn’t a hard prerequisite for many authoring options • Even OS-specific dev technologies (see Docker) are going cross-platform (ish) Writing
11. Writing CLI • + ◦ Lightweight ◦ Composable • - ◦ Features like refactoring ◦ Learning curve IDE • + ◦ Featureful ◦ Tight language integration • - ◦ Complexity ◦ Obfuscation To IDE or not IDE?
12. Writing CLI IDE To IDE or not IDE? Eclipse Atom Language-specific (pycharm, rubymine, etc.) Vim Emacs nano… I guess?
13. Writing Vim • Verb-like editing • Standard • Revitalized with Neovim • Diverse range of language plugins • Arcane for beginners • Feat. like refactoring = poor
14. Writing Emacs • Strong, pluggable codebase • Hotkey-driven • Equally diverse plugin community as Vim • Added functionality (IRC, etc.) • Arcane for beginners • Feat. like refactoring = poor
15. Writing Eclipse • Tight language integration • Pluggable • Integrates SCM • Strong support for langs like Java • Can become resource hog • Hides details under the covers
16. Writing Atom • Latest addition, .js based • Somewhere between CLI and IDE load • Sponsored + Developed at Github • +/- Javascript
17. Writing Traditional Tools • SVN, CVS ◦ Remote SCM server ◦ Distributed development happens via locking ◦ Only relevant pieces checked out Modern Replacements • Git, Mercurial ◦ Server + local components separate ◦ Lightweight branching ◦ Distributed development ◦ De facto OSS winner Source Control
18. Writing Source Control
19. Writing Git, Mercurial • Immutable and graph-like history • Interface options ◦ IDE, CLI, dedicated software • Git vs. Mercurial? ◦ git: low-level swiss army knife ◦ hg: more user-friendly, less adoption
20. Writing Git, Mercurial • Native tools aren’t bad • CLI / IDEs create identical histories; users can collaborate with either • It’s worthwhile to brush up on the data model to avoid major blowups (Google “the git parable”)
21. Utilities
22. Language + Editor + SCM + … ? • Additional command-line tools? ◦ Homebrew, chocolatey, $package_manager • Shell proficiency ◦ Bash is standard, but beefier options are available ◦ Zsh, fish Utilities
23. Language + Editor + SCM + … ? • Language version manager ◦ Versions differ significantly, many projects require a specific major version Utilities jenv, sdkman nvm Rvm, rbenv virtualenv stack … ?
24. Services
25. Services Why? Where do you: • Test the software? • Build the software? • Run the software? • Document the software? • Collaborate on the software? The sad reality: there is no free lunch
26. Services A Word of Warning:
27. Services Collaboration - Cloud GitHub • De facto OSS repository • Git (no mercurial) • Model is pricing for private repos Bitbucket • Smaller but also popular • git + mercurial
28. Services Collaboration - Self-Hosted GitLab • Self-hosted GitHub alternative gogs • Self-contained go service gitolite • CLI-based service
29. Services Testing $ rspec # ruby $ py.test # python $ go test # go ...many more
30. Services Testing Travis CI (Cloud) • Free for OSS • Tightly integrated into GitHub Jenkins (Self-hosted) • Most common self-hosted option
31. Services Testing
32. ?
33. Databases Services
34. Databases Services 1. Don’t try and solve Google-sized scaling issues first 2. SQL isn’t uncool 3. It’s all about the shape of your data 4. Know your access patterns 5. SAFETY 6. ??? 7. Profit
35. Databases Services • SQL ◦ Postgresql, MySQL ◦ Not hip, but been around a long time ◦ Why are they still relevant? ▪ Lots of data is relational! ◦ SQLite convenient option for local development ◦ Well-documented for self-hosting or hosted (RDS) solutions
36. Databases Services • Search ◦ Lucene is the grandaddy ◦ Solr, Elasticsearch ◦ RESTful, JSON-oriented, distributed ◦ Useful to make an application: ▪ Autocomplete ▪ Highlight ▪ Real fast
37. Databases Services • Timeseries ◦ For when you’ve just got numbers ◦ Tuned to numerics
38. Databases Services • Other NoSQL ◦ MongoDB ▪ Easy to get started, scalable ◦ Hadoop Ecosystem ▪ For really, REALLY big data ▪ HDFS, MapReduce, etc. ◦ + many more
39. This is just the surface
40. Questions?
41. Thank You! • GitHub / Speakerdeck ◦ tylerjl ◦ (vim configs at tylerjl/dotfiles on github) • IRC / Twitter ◦ leothrix • tjll.net