Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Everyday Programming for Everyday Problems

Everyday Programming for Everyday Problems

Tyler L

June 10, 2017
Tweet

More Decks by Tyler L

Other Decks in Programming

Transcript

  1. Everyday Programming for
    Everyday Problems
    SouthEast LinuxFest 2017
    Tyler Langlois
    Engineer @ Elastic

    View Slide

  2. $ whois tylerjl
    ● Infrastructure
    engineering/devops at Elastic
    ○ Elasticsearch puppet
    module
    ○ Lots of web properties
    ○ Shell jockey
    ● Other interests
    ○ Linux package maintainer,
    functional programming, my
    dog Morty

    View Slide

  3. Software development is this...

    View Slide

  4. ...but it’s also this

    View Slide

  5. Not all software needs to be over engineered

    View Slide

  6. Large-scale problems are different than small
    problems

    View Slide

  7. Introduction
    Some actual small-scale solutions

    View Slide

  8. High-Level Tools
    ● Ruby, Python, Node,
    etc. all have useful,
    expansive libraries
    ● Plenty of open and
    free APIs
    Why This is Easier Than it’s Ever Been
    Cheap Computing
    ● VPS providers offer
    cheap, easy-to-use
    Linux servers
    ● Raspberry Pi/other
    SBCs
    Pre-built Frontends
    ● Chat bots make for
    easy interfaces
    ● Lots of web tools
    (wikis, dashboards
    like Kibana, etc.)

    View Slide

  9. Case Studies

    View Slide

  10. Case Study: The Angry Tweeter

    View Slide

  11. Case Study: The Angry Tweeter
    Before:

    View Slide

  12. Case Study: The Angry Tweeter
    After:

    View Slide

  13. Case Study: The Angry Tweeter
    Why?
    ● Measurable: verify that paid speeds are being met
    ● Recorded: proof of historical bandwidth provided
    ● Accurate: “internet is slow” != ISP behaving badly

    View Slide

  14. Case Study: The Angry Tweeter
    How?
    Answer the relevant questions:
    ● How will you measure bandwidth?
    ○ Google: “python bandwidth test” (first hit)

    View Slide

  15. Case Study: The Angry Tweeter
    How?
    Answer the relevant questions:
    ● Where will you put the measured bandwidth?
    ○ Storing lots of numbers (timeseries data) is its own can of worms
    ○ tl;dr whatever works for you
    graphite, ganglia, elasticsearch,
    kairosdb, influxdb, prometheus,
    hawkular, etc….

    View Slide

  16. Case Study: The Angry Tweeter
    How?
    Answer the relevant questions:
    ● How will you view your bandwidth?
    ○ Largely dependent on where you store your bandwidth
    ○ Tried and true solutions like Kibana/Grafana are good options

    View Slide

  17. Case Study: The Angry Tweeter
    How?
    Just some python, plus some plumbing:
    from pyspeedtest import SpeedTest
    from elasticsearch import Elasticsearch
    tester = SpeedTest()
    es = Elasticsearch()
    while sleep(900):
    es.index(index=’bw’, type=’bw’, body={‘down’:tester.download()})

    View Slide

  18. Case Study: The Angry Tweeter
    Results

    View Slide

  19. Case Study: The Angry Tweeter
    Results

    View Slide

  20. Case Study: Printers (╯°□°)╯︵ ┻━┻

    View Slide

  21. Case Study: Printers (╯°□°)╯︵ ┻━┻
    My Typical Workflow

    View Slide

  22. View Slide

  23. Case Study: Printers (╯°□°)╯︵ ┻━┻
    My Typical Workflow

    View Slide

  24. Case Study: Printers (╯°□°)╯︵ ┻━┻
    Alternative: Build It Yourself!

    View Slide

  25. Case Study: Printers (╯°□°)╯︵ ┻━┻
    Alternative: Build It Yourself!
    ● Print server
    ○ Raspberry Pi + cups
    ● Wireless printing (smartphones, too!)
    ○ Avahi + 1 config file
    Zero client config, secure, configurable,
    stable

    View Slide

  26. Case Study: Bots

    View Slide

  27. Case Study: Bots
    Why?
    ● Low barrier to entry for simple UX
    ● Very high-level libraries
    ● Slack makes private chat channels easy
    (i.e., no IRC bouncer)
    ● Code in whatever you want!

    View Slide

  28. Case Study: Bots
    Simple commands

    View Slide

  29. Case Study: Bots
    Simple commands
    Easy to make a chat interface for anything you can come up with:
    require 'slack-ruby-bot'
    class MyBot < SlackRubyBot::Bot
    command ‘day’ do |client, data, match|
    client.say(text: Time.now.strftime('%A'), channel: data.channel)
    end
    end
    MyBot.run

    View Slide

  30. Case Study: Media Archiving

    View Slide

  31. Case Study: Media Archiving
    ?

    View Slide

  32. Case Study: Media Archiving
    Options
    ● By hand
    ○ Handbrake buttons + copy/paste + etc. = waaaay too long
    ● Buy digital copies
    ○ Haha
    ● Send them somewhere for conversion
    ○ I’m not about to admit defeat

    View Slide

  33. Case Study: Media Archiving
    Solution
    dev-cdrom.device
    autorip.service
    DVD.mkv
    $ eject /dev/cdrom

    View Slide

  34. Case Study: Media Archiving
    Results
    ● Entire process is:
    ○ Insert DVD, remove DVD after ~1 hour
    ○ Media handling is another system (NAS + Kodi)
    ● In total, maybe 50 lines of code and config files
    ○ Just shell scripts + systemd configs
    ● Custom format, compression, subtitles, etc.
    ● Event flow is easy to grok with systemd

    View Slide

  35. Case Studies: So Many More!

    View Slide

  36. Considerations

    View Slide

  37. ● Whatever gels with your
    brain
    ● Ruby/Python/Node/Perl are
    all high-level languages with
    powerful libraries
    ● Run it on Linux?
    ○ Easy scheduling
    ○ Cheap servers
    ○ Strongest tooling
    Considerations:
    ● What do I use?
    ● How do I write it?
    ● Where do I run it?

    View Slide

  38. ● Try starting low-level to
    understand, then move up
    (i.e., command line)
    ● At least an editor
    (vim/emacs/atom)
    ● IDEs can be extra helpful
    (PyCharm/Rubymine/
    Eclipse/etc.)
    Considerations:
    ● What do I use?
    ● How do I write it?
    ● Where do I run it?

    View Slide

  39. Considerations:
    ● What do I use?
    ● How do I write it?
    ● Where do I run it?
    ● One time?
    ○ Just run it locally
    ● All the time?
    ○ Any computer that runs
    all the time
    ● All-in?
    ○ Little Linux server
    ○ (this is easier and safer
    than you think)
    ○ Or, just buy a VPS
    ● Self-hosting is now sub-$25

    View Slide

  40. Where to go From Here?
    ● Zero programming experience?
    ○ Get started with a tutorial about $x
    ● Looking for something to solve?
    ○ Pick something annoying
    ● Can’t figure out how to solve it?
    ○ Lucky you, you’re at SELF
    ● Solved something cool?
    ○ Blog about it and tell us!

    View Slide

  41. Thank you!
    github.com/tylerjl
    irc/twitter: leothrix
    tjll.net
    Additional Information:
    ● speakerdeck.com/tylerjl
    ● Corner me anytime this
    conference
    ○ (I’ll be at the Elastic
    booth after)

    View Slide