Everyday Programming for Everyday Problems

Everyday Programming for Everyday Problems

F43919144cdcddd7ba50e46f71667d08?s=128

Tyler L

June 10, 2017
Tweet

Transcript

  1. Everyday Programming for Everyday Problems SouthEast LinuxFest 2017 Tyler Langlois

    Engineer @ Elastic
  2. $ whois tylerjl • Infrastructure engineering/devops at Elastic ◦ Elasticsearch

    puppet module ◦ Lots of web properties ◦ Shell jockey • Other interests ◦ Linux package maintainer, functional programming, my dog Morty
  3. Software development is this...

  4. ...but it’s also this

  5. Not all software needs to be over engineered

  6. Large-scale problems are different than small problems

  7. Introduction Some actual small-scale solutions

  8. High-Level Tools • Ruby, Python, Node, etc. all have useful,

    expansive libraries • Plenty of open and free APIs Why This is Easier Than it’s Ever Been Cheap Computing • VPS providers offer cheap, easy-to-use Linux servers • Raspberry Pi/other SBCs Pre-built Frontends • Chat bots make for easy interfaces • Lots of web tools (wikis, dashboards like Kibana, etc.)
  9. Case Studies

  10. Case Study: The Angry Tweeter

  11. Case Study: The Angry Tweeter Before:

  12. Case Study: The Angry Tweeter After:

  13. Case Study: The Angry Tweeter Why? • Measurable: verify that

    paid speeds are being met • Recorded: proof of historical bandwidth provided • Accurate: “internet is slow” != ISP behaving badly
  14. Case Study: The Angry Tweeter How? Answer the relevant questions:

    • How will you measure bandwidth? ◦ Google: “python bandwidth test” (first hit)
  15. Case Study: The Angry Tweeter How? Answer the relevant questions:

    • Where will you put the measured bandwidth? ◦ Storing lots of numbers (timeseries data) is its own can of worms ◦ tl;dr whatever works for you graphite, ganglia, elasticsearch, kairosdb, influxdb, prometheus, hawkular, etc….
  16. Case Study: The Angry Tweeter How? Answer the relevant questions:

    • How will you view your bandwidth? ◦ Largely dependent on where you store your bandwidth ◦ Tried and true solutions like Kibana/Grafana are good options
  17. Case Study: The Angry Tweeter How? Just some python, plus

    some plumbing: from pyspeedtest import SpeedTest from elasticsearch import Elasticsearch tester = SpeedTest() es = Elasticsearch() while sleep(900): es.index(index=’bw’, type=’bw’, body={‘down’:tester.download()})
  18. Case Study: The Angry Tweeter Results

  19. Case Study: The Angry Tweeter Results

  20. Case Study: Printers (╯°□°)╯︵ ┻━┻

  21. Case Study: Printers (╯°□°)╯︵ ┻━┻ My Typical Workflow

  22. None
  23. Case Study: Printers (╯°□°)╯︵ ┻━┻ My Typical Workflow

  24. Case Study: Printers (╯°□°)╯︵ ┻━┻ Alternative: Build It Yourself!

  25. Case Study: Printers (╯°□°)╯︵ ┻━┻ Alternative: Build It Yourself! •

    Print server ◦ Raspberry Pi + cups • Wireless printing (smartphones, too!) ◦ Avahi + 1 config file Zero client config, secure, configurable, stable
  26. Case Study: Bots

  27. Case Study: Bots Why? • Low barrier to entry for

    simple UX • Very high-level libraries • Slack makes private chat channels easy (i.e., no IRC bouncer) • Code in whatever you want!
  28. Case Study: Bots Simple commands

  29. Case Study: Bots Simple commands Easy to make a chat

    interface for anything you can come up with: require 'slack-ruby-bot' class MyBot < SlackRubyBot::Bot command ‘day’ do |client, data, match| client.say(text: Time.now.strftime('%A'), channel: data.channel) end end MyBot.run
  30. Case Study: Media Archiving

  31. Case Study: Media Archiving ?

  32. Case Study: Media Archiving Options • By hand ◦ Handbrake

    buttons + copy/paste + etc. = waaaay too long • Buy digital copies ◦ Haha • Send them somewhere for conversion ◦ I’m not about to admit defeat
  33. Case Study: Media Archiving Solution dev-cdrom.device autorip.service DVD.mkv $ eject

    /dev/cdrom
  34. Case Study: Media Archiving Results • Entire process is: ◦

    Insert DVD, remove DVD after ~1 hour ◦ Media handling is another system (NAS + Kodi) • In total, maybe 50 lines of code and config files ◦ Just shell scripts + systemd configs • Custom format, compression, subtitles, etc. • Event flow is easy to grok with systemd
  35. Case Studies: So Many More!

  36. Considerations

  37. • Whatever gels with your brain • Ruby/Python/Node/Perl are all

    high-level languages with powerful libraries • Run it on Linux? ◦ Easy scheduling ◦ Cheap servers ◦ Strongest tooling Considerations: • What do I use? • How do I write it? • Where do I run it?
  38. • Try starting low-level to understand, then move up (i.e.,

    command line) • At least an editor (vim/emacs/atom) • IDEs can be extra helpful (PyCharm/Rubymine/ Eclipse/etc.) Considerations: • What do I use? • How do I write it? • Where do I run it?
  39. Considerations: • What do I use? • How do I

    write it? • Where do I run it? • One time? ◦ Just run it locally • All the time? ◦ Any computer that runs all the time • All-in? ◦ Little Linux server ◦ (this is easier and safer than you think) ◦ Or, just buy a VPS • Self-hosting is now sub-$25
  40. Where to go From Here? • Zero programming experience? ◦

    Get started with a tutorial about $x • Looking for something to solve? ◦ Pick something annoying • Can’t figure out how to solve it? ◦ Lucky you, you’re at SELF • Solved something cool? ◦ Blog about it and tell us!
  41. Thank you! github.com/tylerjl irc/twitter: leothrix tjll.net Additional Information: • speakerdeck.com/tylerjl

    • Corner me anytime this conference ◦ (I’ll be at the Elastic booth after)