Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Everyday Programming for Everyday Problems

Everyday Programming for Everyday Problems

Tyler L

June 10, 2017
Tweet

More Decks by Tyler L

Other Decks in Programming

Transcript

  1. Everyday Programming for
    Everyday Problems
    SouthEast LinuxFest 2017
    Tyler Langlois
    Engineer @ Elastic

    View full-size slide

  2. $ whois tylerjl
    ● Infrastructure
    engineering/devops at Elastic
    ○ Elasticsearch puppet
    module
    ○ Lots of web properties
    ○ Shell jockey
    ● Other interests
    ○ Linux package maintainer,
    functional programming, my
    dog Morty

    View full-size slide

  3. Software development is this...

    View full-size slide

  4. ...but it’s also this

    View full-size slide

  5. Not all software needs to be over engineered

    View full-size slide

  6. Large-scale problems are different than small
    problems

    View full-size slide

  7. Introduction
    Some actual small-scale solutions

    View full-size slide

  8. High-Level Tools
    ● Ruby, Python, Node,
    etc. all have useful,
    expansive libraries
    ● Plenty of open and
    free APIs
    Why This is Easier Than it’s Ever Been
    Cheap Computing
    ● VPS providers offer
    cheap, easy-to-use
    Linux servers
    ● Raspberry Pi/other
    SBCs
    Pre-built Frontends
    ● Chat bots make for
    easy interfaces
    ● Lots of web tools
    (wikis, dashboards
    like Kibana, etc.)

    View full-size slide

  9. Case Studies

    View full-size slide

  10. Case Study: The Angry Tweeter

    View full-size slide

  11. Case Study: The Angry Tweeter
    Before:

    View full-size slide

  12. Case Study: The Angry Tweeter
    After:

    View full-size slide

  13. Case Study: The Angry Tweeter
    Why?
    ● Measurable: verify that paid speeds are being met
    ● Recorded: proof of historical bandwidth provided
    ● Accurate: “internet is slow” != ISP behaving badly

    View full-size slide

  14. Case Study: The Angry Tweeter
    How?
    Answer the relevant questions:
    ● How will you measure bandwidth?
    ○ Google: “python bandwidth test” (first hit)

    View full-size slide

  15. Case Study: The Angry Tweeter
    How?
    Answer the relevant questions:
    ● Where will you put the measured bandwidth?
    ○ Storing lots of numbers (timeseries data) is its own can of worms
    ○ tl;dr whatever works for you
    graphite, ganglia, elasticsearch,
    kairosdb, influxdb, prometheus,
    hawkular, etc….

    View full-size slide

  16. Case Study: The Angry Tweeter
    How?
    Answer the relevant questions:
    ● How will you view your bandwidth?
    ○ Largely dependent on where you store your bandwidth
    ○ Tried and true solutions like Kibana/Grafana are good options

    View full-size slide

  17. Case Study: The Angry Tweeter
    How?
    Just some python, plus some plumbing:
    from pyspeedtest import SpeedTest
    from elasticsearch import Elasticsearch
    tester = SpeedTest()
    es = Elasticsearch()
    while sleep(900):
    es.index(index=’bw’, type=’bw’, body={‘down’:tester.download()})

    View full-size slide

  18. Case Study: The Angry Tweeter
    Results

    View full-size slide

  19. Case Study: The Angry Tweeter
    Results

    View full-size slide

  20. Case Study: Printers (╯°□°)╯︵ ┻━┻

    View full-size slide

  21. Case Study: Printers (╯°□°)╯︵ ┻━┻
    My Typical Workflow

    View full-size slide

  22. Case Study: Printers (╯°□°)╯︵ ┻━┻
    My Typical Workflow

    View full-size slide

  23. Case Study: Printers (╯°□°)╯︵ ┻━┻
    Alternative: Build It Yourself!

    View full-size slide

  24. Case Study: Printers (╯°□°)╯︵ ┻━┻
    Alternative: Build It Yourself!
    ● Print server
    ○ Raspberry Pi + cups
    ● Wireless printing (smartphones, too!)
    ○ Avahi + 1 config file
    Zero client config, secure, configurable,
    stable

    View full-size slide

  25. Case Study: Bots

    View full-size slide

  26. Case Study: Bots
    Why?
    ● Low barrier to entry for simple UX
    ● Very high-level libraries
    ● Slack makes private chat channels easy
    (i.e., no IRC bouncer)
    ● Code in whatever you want!

    View full-size slide

  27. Case Study: Bots
    Simple commands

    View full-size slide

  28. Case Study: Bots
    Simple commands
    Easy to make a chat interface for anything you can come up with:
    require 'slack-ruby-bot'
    class MyBot < SlackRubyBot::Bot
    command ‘day’ do |client, data, match|
    client.say(text: Time.now.strftime('%A'), channel: data.channel)
    end
    end
    MyBot.run

    View full-size slide

  29. Case Study: Media Archiving

    View full-size slide

  30. Case Study: Media Archiving
    ?

    View full-size slide

  31. Case Study: Media Archiving
    Options
    ● By hand
    ○ Handbrake buttons + copy/paste + etc. = waaaay too long
    ● Buy digital copies
    ○ Haha
    ● Send them somewhere for conversion
    ○ I’m not about to admit defeat

    View full-size slide

  32. Case Study: Media Archiving
    Solution
    dev-cdrom.device
    autorip.service
    DVD.mkv
    $ eject /dev/cdrom

    View full-size slide

  33. Case Study: Media Archiving
    Results
    ● Entire process is:
    ○ Insert DVD, remove DVD after ~1 hour
    ○ Media handling is another system (NAS + Kodi)
    ● In total, maybe 50 lines of code and config files
    ○ Just shell scripts + systemd configs
    ● Custom format, compression, subtitles, etc.
    ● Event flow is easy to grok with systemd

    View full-size slide

  34. Case Studies: So Many More!

    View full-size slide

  35. Considerations

    View full-size slide

  36. ● Whatever gels with your
    brain
    ● Ruby/Python/Node/Perl are
    all high-level languages with
    powerful libraries
    ● Run it on Linux?
    ○ Easy scheduling
    ○ Cheap servers
    ○ Strongest tooling
    Considerations:
    ● What do I use?
    ● How do I write it?
    ● Where do I run it?

    View full-size slide

  37. ● Try starting low-level to
    understand, then move up
    (i.e., command line)
    ● At least an editor
    (vim/emacs/atom)
    ● IDEs can be extra helpful
    (PyCharm/Rubymine/
    Eclipse/etc.)
    Considerations:
    ● What do I use?
    ● How do I write it?
    ● Where do I run it?

    View full-size slide

  38. Considerations:
    ● What do I use?
    ● How do I write it?
    ● Where do I run it?
    ● One time?
    ○ Just run it locally
    ● All the time?
    ○ Any computer that runs
    all the time
    ● All-in?
    ○ Little Linux server
    ○ (this is easier and safer
    than you think)
    ○ Or, just buy a VPS
    ● Self-hosting is now sub-$25

    View full-size slide

  39. Where to go From Here?
    ● Zero programming experience?
    ○ Get started with a tutorial about $x
    ● Looking for something to solve?
    ○ Pick something annoying
    ● Can’t figure out how to solve it?
    ○ Lucky you, you’re at SELF
    ● Solved something cool?
    ○ Blog about it and tell us!

    View full-size slide

  40. Thank you!
    github.com/tylerjl
    irc/twitter: leothrix
    tjll.net
    Additional Information:
    ● speakerdeck.com/tylerjl
    ● Corner me anytime this
    conference
    ○ (I’ll be at the Elastic
    booth after)

    View full-size slide