Everyday Programming for Everyday Problems

Transcript

1. Everyday Programming for
   Everyday Problems
   SouthEast LinuxFest 2017
   Tyler Langlois
   Engineer @ Elastic
   

   View Slide

2. $ whois tylerjl
   ● Infrastructure
   engineering/devops at Elastic
   ○ Elasticsearch puppet
   module
   ○ Lots of web properties
   ○ Shell jockey
   ● Other interests
   ○ Linux package maintainer,
   functional programming, my
   dog Morty
   

   View Slide

3. Software development is this...
   

   View Slide

4. ...but it’s also this
   

   View Slide

5. Not all software needs to be over engineered
   

   View Slide

6. Large-scale problems are different than small
   problems
   

   View Slide

7. Introduction
   Some actual small-scale solutions
   

   View Slide

8. High-Level Tools
   ● Ruby, Python, Node,
   etc. all have useful,
   expansive libraries
   ● Plenty of open and
   free APIs
   Why This is Easier Than it’s Ever Been
   Cheap Computing
   ● VPS providers offer
   cheap, easy-to-use
   Linux servers
   ● Raspberry Pi/other
   SBCs
   Pre-built Frontends
   ● Chat bots make for
   easy interfaces
   ● Lots of web tools
   (wikis, dashboards
   like Kibana, etc.)
   

   View Slide

9. Case Studies
   

   View Slide

10. Case Study: The Angry Tweeter
    

    View Slide

11. Case Study: The Angry Tweeter
    Before:
    

    View Slide

12. Case Study: The Angry Tweeter
    After:
    

    View Slide

13. Case Study: The Angry Tweeter
    Why?
    ● Measurable: verify that paid speeds are being met
    ● Recorded: proof of historical bandwidth provided
    ● Accurate: “internet is slow” != ISP behaving badly
    

    View Slide

14. Case Study: The Angry Tweeter
    How?
    Answer the relevant questions:
    ● How will you measure bandwidth?
    ○ Google: “python bandwidth test” (first hit)
    

    View Slide

15. Case Study: The Angry Tweeter
    How?
    Answer the relevant questions:
    ● Where will you put the measured bandwidth?
    ○ Storing lots of numbers (timeseries data) is its own can of worms
    ○ tl;dr whatever works for you
    graphite, ganglia, elasticsearch,
    kairosdb, influxdb, prometheus,
    hawkular, etc….
    

    View Slide

16. Case Study: The Angry Tweeter
    How?
    Answer the relevant questions:
    ● How will you view your bandwidth?
    ○ Largely dependent on where you store your bandwidth
    ○ Tried and true solutions like Kibana/Grafana are good options
    

    View Slide

17. Case Study: The Angry Tweeter
    How?
    Just some python, plus some plumbing:
    from pyspeedtest import SpeedTest
    from elasticsearch import Elasticsearch
    tester = SpeedTest()
    es = Elasticsearch()
    while sleep(900):
    es.index(index=’bw’, type=’bw’, body={‘down’:tester.download()})
    

    View Slide

18. Case Study: The Angry Tweeter
    Results
    

    View Slide

19. Case Study: The Angry Tweeter
    Results
    

    View Slide

20. Case Study: Printers (╯°□°）╯︵ ┻━┻
    

    View Slide

21. Case Study: Printers (╯°□°）╯︵ ┻━┻
    My Typical Workflow
    

    View Slide

22. View Slide

23. Case Study: Printers (╯°□°）╯︵ ┻━┻
    My Typical Workflow
    

    View Slide

24. Case Study: Printers (╯°□°）╯︵ ┻━┻
    Alternative: Build It Yourself!
    

    View Slide

25. Case Study: Printers (╯°□°）╯︵ ┻━┻
    Alternative: Build It Yourself!
    ● Print server
    ○ Raspberry Pi + cups
    ● Wireless printing (smartphones, too!)
    ○ Avahi + 1 config file
    Zero client config, secure, configurable,
    stable
    

    View Slide

26. Case Study: Bots
    

    View Slide

27. Case Study: Bots
    Why?
    ● Low barrier to entry for simple UX
    ● Very high-level libraries
    ● Slack makes private chat channels easy
    (i.e., no IRC bouncer)
    ● Code in whatever you want!
    

    View Slide

28. Case Study: Bots
    Simple commands
    

    View Slide

29. Case Study: Bots
    Simple commands
    Easy to make a chat interface for anything you can come up with:
    require 'slack-ruby-bot'
    class MyBot < SlackRubyBot::Bot
    command ‘day’ do |client, data, match|
    client.say(text: Time.now.strftime('%A'), channel: data.channel)
    end
    end
    MyBot.run
    

    View Slide

30. Case Study: Media Archiving
    

    View Slide

31. Case Study: Media Archiving
    ?
    

    View Slide

32. Case Study: Media Archiving
    Options
    ● By hand
    ○ Handbrake buttons + copy/paste + etc. = waaaay too long
    ● Buy digital copies
    ○ Haha
    ● Send them somewhere for conversion
    ○ I’m not about to admit defeat
    

    View Slide

33. Case Study: Media Archiving
    Solution
    dev-cdrom.device
    autorip.service
    DVD.mkv
    $ eject /dev/cdrom
    

    View Slide

34. Case Study: Media Archiving
    Results
    ● Entire process is:
    ○ Insert DVD, remove DVD after ~1 hour
    ○ Media handling is another system (NAS + Kodi)
    ● In total, maybe 50 lines of code and config files
    ○ Just shell scripts + systemd configs
    ● Custom format, compression, subtitles, etc.
    ● Event flow is easy to grok with systemd
    

    View Slide

35. Case Studies: So Many More!
    

    View Slide

36. Considerations
    

    View Slide

37. ● Whatever gels with your
    brain
    ● Ruby/Python/Node/Perl are
    all high-level languages with
    powerful libraries
    ● Run it on Linux?
    ○ Easy scheduling
    ○ Cheap servers
    ○ Strongest tooling
    Considerations:
    ● What do I use?
    ● How do I write it?
    ● Where do I run it?
    

    View Slide

38. ● Try starting low-level to
    understand, then move up
    (i.e., command line)
    ● At least an editor
    (vim/emacs/atom)
    ● IDEs can be extra helpful
    (PyCharm/Rubymine/
    Eclipse/etc.)
    Considerations:
    ● What do I use?
    ● How do I write it?
    ● Where do I run it?
    

    View Slide

39. Considerations:
    ● What do I use?
    ● How do I write it?
    ● Where do I run it?
    ● One time?
    ○ Just run it locally
    ● All the time?
    ○ Any computer that runs
    all the time
    ● All-in?
    ○ Little Linux server
    ○ (this is easier and safer
    than you think)
    ○ Or, just buy a VPS
    ● Self-hosting is now sub-$25
    

    View Slide

40. Where to go From Here?
    ● Zero programming experience?
    ○ Get started with a tutorial about $x
    ● Looking for something to solve?
    ○ Pick something annoying
    ● Can’t figure out how to solve it?
    ○ Lucky you, you’re at SELF
    ● Solved something cool?
    ○ Blog about it and tell us!
    

    View Slide

41. Thank you!
    github.com/tylerjl
    irc/twitter: leothrix
    tjll.net
    Additional Information:
    ● speakerdeck.com/tylerjl
    ● Corner me anytime this
    conference
    ○ (I’ll be at the Elastic
    booth after)
    

    View Slide