Everyday Programming for Everyday Problems

Transcript

1. Everyday Programming for Everyday Problems SouthEast LinuxFest 2017 Tyler Langlois Engineer @ Elastic
2. $ whois tylerjl • Infrastructure engineering/devops at Elastic ◦ Elasticsearch puppet module ◦ Lots of web properties ◦ Shell jockey • Other interests ◦ Linux package maintainer, functional programming, my dog Morty
3. Software development is this...
4. ...but it’s also this
5. Not all software needs to be over engineered
6. Large-scale problems are different than small problems
7. Introduction Some actual small-scale solutions
8. High-Level Tools • Ruby, Python, Node, etc. all have useful, expansive libraries • Plenty of open and free APIs Why This is Easier Than it’s Ever Been Cheap Computing • VPS providers offer cheap, easy-to-use Linux servers • Raspberry Pi/other SBCs Pre-built Frontends • Chat bots make for easy interfaces • Lots of web tools (wikis, dashboards like Kibana, etc.)
9. Case Studies
10. Case Study: The Angry Tweeter
11. Case Study: The Angry Tweeter Before:
12. Case Study: The Angry Tweeter After:
13. Case Study: The Angry Tweeter Why? • Measurable: verify that paid speeds are being met • Recorded: proof of historical bandwidth provided • Accurate: “internet is slow” != ISP behaving badly
14. Case Study: The Angry Tweeter How? Answer the relevant questions: • How will you measure bandwidth? ◦ Google: “python bandwidth test” (first hit)
15. Case Study: The Angry Tweeter How? Answer the relevant questions: • Where will you put the measured bandwidth? ◦ Storing lots of numbers (timeseries data) is its own can of worms ◦ tl;dr whatever works for you graphite, ganglia, elasticsearch, kairosdb, influxdb, prometheus, hawkular, etc….
16. Case Study: The Angry Tweeter How? Answer the relevant questions: • How will you view your bandwidth? ◦ Largely dependent on where you store your bandwidth ◦ Tried and true solutions like Kibana/Grafana are good options
17. Case Study: The Angry Tweeter How? Just some python, plus some plumbing: from pyspeedtest import SpeedTest from elasticsearch import Elasticsearch tester = SpeedTest() es = Elasticsearch() while sleep(900): es.index(index=’bw’, type=’bw’, body={‘down’:tester.download()})
18. Case Study: The Angry Tweeter Results
19. Case Study: The Angry Tweeter Results
20. Case Study: Printers (╯°□°）╯︵ ┻━┻
21. Case Study: Printers (╯°□°）╯︵ ┻━┻ My Typical Workflow
23. Case Study: Printers (╯°□°）╯︵ ┻━┻ My Typical Workflow
24. Case Study: Printers (╯°□°）╯︵ ┻━┻ Alternative: Build It Yourself!
25. Case Study: Printers (╯°□°）╯︵ ┻━┻ Alternative: Build It Yourself! • Print server ◦ Raspberry Pi + cups • Wireless printing (smartphones, too!) ◦ Avahi + 1 config file Zero client config, secure, configurable, stable
26. Case Study: Bots
27. Case Study: Bots Why? • Low barrier to entry for simple UX • Very high-level libraries • Slack makes private chat channels easy (i.e., no IRC bouncer) • Code in whatever you want!
28. Case Study: Bots Simple commands
29. Case Study: Bots Simple commands Easy to make a chat interface for anything you can come up with: require 'slack-ruby-bot' class MyBot < SlackRubyBot::Bot command ‘day’ do |client, data, match| client.say(text: Time.now.strftime('%A'), channel: data.channel) end end MyBot.run
30. Case Study: Media Archiving
31. Case Study: Media Archiving ?
32. Case Study: Media Archiving Options • By hand ◦ Handbrake buttons + copy/paste + etc. = waaaay too long • Buy digital copies ◦ Haha • Send them somewhere for conversion ◦ I’m not about to admit defeat
33. Case Study: Media Archiving Solution dev-cdrom.device autorip.service DVD.mkv $ eject /dev/cdrom
34. Case Study: Media Archiving Results • Entire process is: ◦ Insert DVD, remove DVD after ~1 hour ◦ Media handling is another system (NAS + Kodi) • In total, maybe 50 lines of code and config files ◦ Just shell scripts + systemd configs • Custom format, compression, subtitles, etc. • Event flow is easy to grok with systemd
35. Case Studies: So Many More!
36. Considerations
37. • Whatever gels with your brain • Ruby/Python/Node/Perl are all high-level languages with powerful libraries • Run it on Linux? ◦ Easy scheduling ◦ Cheap servers ◦ Strongest tooling Considerations: • What do I use? • How do I write it? • Where do I run it?
38. • Try starting low-level to understand, then move up (i.e., command line) • At least an editor (vim/emacs/atom) • IDEs can be extra helpful (PyCharm/Rubymine/ Eclipse/etc.) Considerations: • What do I use? • How do I write it? • Where do I run it?
39. Considerations: • What do I use? • How do I write it? • Where do I run it? • One time? ◦ Just run it locally • All the time? ◦ Any computer that runs all the time • All-in? ◦ Little Linux server ◦ (this is easier and safer than you think) ◦ Or, just buy a VPS • Self-hosting is now sub-$25
40. Where to go From Here? • Zero programming experience? ◦ Get started with a tutorial about $x • Looking for something to solve? ◦ Pick something annoying • Can’t figure out how to solve it? ◦ Lucky you, you’re at SELF • Solved something cool? ◦ Blog about it and tell us!
41. Thank you! github.com/tylerjl irc/twitter: leothrix tjll.net Additional Information: • speakerdeck.com/tylerjl • Corner me anytime this conference ◦ (I’ll be at the Elastic booth after)