Tweet
Tweet

Slide 1

Slide 1 text

Everyday Programming for Everyday Problems SouthEast LinuxFest 2017 Tyler Langlois Engineer @ Elastic

Slide 2

Slide 2 text

$ whois tylerjl ● Infrastructure engineering/devops at Elastic ○ Elasticsearch puppet module ○ Lots of web properties ○ Shell jockey ● Other interests ○ Linux package maintainer, functional programming, my dog Morty

Slide 3

Slide 3 text

Software development is this...

Slide 4

Slide 4 text

...but it’s also this

Slide 5

Slide 5 text

Not all software needs to be over engineered

Slide 6

Slide 6 text

Large-scale problems are different than small problems

Slide 7

Slide 7 text

Introduction Some actual small-scale solutions

Slide 8

Slide 8 text

High-Level Tools ● Ruby, Python, Node, etc. all have useful, expansive libraries ● Plenty of open and free APIs Why This is Easier Than it’s Ever Been Cheap Computing ● VPS providers offer cheap, easy-to-use Linux servers ● Raspberry Pi/other SBCs Pre-built Frontends ● Chat bots make for easy interfaces ● Lots of web tools (wikis, dashboards like Kibana, etc.)

Slide 9

Slide 9 text

Case Studies

Slide 10

Slide 10 text

Case Study: The Angry Tweeter

Slide 11

Slide 11 text

Case Study: The Angry Tweeter Before:

Slide 12

Slide 12 text

Case Study: The Angry Tweeter After:

Slide 13

Slide 13 text

Case Study: The Angry Tweeter Why? ● Measurable: verify that paid speeds are being met ● Recorded: proof of historical bandwidth provided ● Accurate: “internet is slow” != ISP behaving badly

Slide 14

Slide 14 text

Case Study: The Angry Tweeter How? Answer the relevant questions: ● How will you measure bandwidth? ○ Google: “python bandwidth test” (first hit)

Slide 15

Slide 15 text

Case Study: The Angry Tweeter How? Answer the relevant questions: ● Where will you put the measured bandwidth? ○ Storing lots of numbers (timeseries data) is its own can of worms ○ tl;dr whatever works for you graphite, ganglia, elasticsearch, kairosdb, influxdb, prometheus, hawkular, etc….

Slide 16

Slide 16 text

Case Study: The Angry Tweeter How? Answer the relevant questions: ● How will you view your bandwidth? ○ Largely dependent on where you store your bandwidth ○ Tried and true solutions like Kibana/Grafana are good options

Slide 17

Slide 17 text

Case Study: The Angry Tweeter How? Just some python, plus some plumbing: from pyspeedtest import SpeedTest from elasticsearch import Elasticsearch tester = SpeedTest() es = Elasticsearch() while sleep(900): es.index(index=’bw’, type=’bw’, body={‘down’:tester.download()})

Slide 18

Slide 18 text

Case Study: The Angry Tweeter Results

Slide 19

Slide 19 text

Case Study: The Angry Tweeter Results

Slide 20

Slide 20 text

Case Study: Printers (╯°□°）╯︵ ┻━┻

Slide 21

Slide 21 text

Case Study: Printers (╯°□°）╯︵ ┻━┻ My Typical Workflow

Slide 22

Slide 22 text

No content

Slide 23

Slide 23 text

Case Study: Printers (╯°□°）╯︵ ┻━┻ My Typical Workflow

Slide 24

Slide 24 text

Case Study: Printers (╯°□°）╯︵ ┻━┻ Alternative: Build It Yourself!

Slide 25

Slide 25 text

Case Study: Printers (╯°□°）╯︵ ┻━┻ Alternative: Build It Yourself! ● Print server ○ Raspberry Pi + cups ● Wireless printing (smartphones, too!) ○ Avahi + 1 config file Zero client config, secure, configurable, stable

Slide 26

Slide 26 text

Case Study: Bots

Slide 27

Slide 27 text

Case Study: Bots Why? ● Low barrier to entry for simple UX ● Very high-level libraries ● Slack makes private chat channels easy (i.e., no IRC bouncer) ● Code in whatever you want!

Slide 28

Slide 28 text

Case Study: Bots Simple commands

Slide 29

Slide 29 text

Case Study: Bots Simple commands Easy to make a chat interface for anything you can come up with: require 'slack-ruby-bot' class MyBot < SlackRubyBot::Bot command ‘day’ do |client, data, match| client.say(text: Time.now.strftime('%A'), channel: data.channel) end end MyBot.run

Slide 30

Slide 30 text

Case Study: Media Archiving

Slide 31

Slide 31 text

Case Study: Media Archiving ?

Slide 32

Slide 32 text

Case Study: Media Archiving Options ● By hand ○ Handbrake buttons + copy/paste + etc. = waaaay too long ● Buy digital copies ○ Haha ● Send them somewhere for conversion ○ I’m not about to admit defeat

Slide 33

Slide 33 text

Case Study: Media Archiving Solution dev-cdrom.device autorip.service DVD.mkv $ eject /dev/cdrom

Slide 34

Slide 34 text

Case Study: Media Archiving Results ● Entire process is: ○ Insert DVD, remove DVD after ~1 hour ○ Media handling is another system (NAS + Kodi) ● In total, maybe 50 lines of code and config files ○ Just shell scripts + systemd configs ● Custom format, compression, subtitles, etc. ● Event flow is easy to grok with systemd

Slide 35

Slide 35 text

Case Studies: So Many More!

Slide 36

Slide 36 text

Considerations

Slide 37

Slide 37 text

● Whatever gels with your brain ● Ruby/Python/Node/Perl are all high-level languages with powerful libraries ● Run it on Linux? ○ Easy scheduling ○ Cheap servers ○ Strongest tooling Considerations: ● What do I use? ● How do I write it? ● Where do I run it?

Slide 38

Slide 38 text

● Try starting low-level to understand, then move up (i.e., command line) ● At least an editor (vim/emacs/atom) ● IDEs can be extra helpful (PyCharm/Rubymine/ Eclipse/etc.) Considerations: ● What do I use? ● How do I write it? ● Where do I run it?

Slide 39

Slide 39 text

Considerations: ● What do I use? ● How do I write it? ● Where do I run it? ● One time? ○ Just run it locally ● All the time? ○ Any computer that runs all the time ● All-in? ○ Little Linux server ○ (this is easier and safer than you think) ○ Or, just buy a VPS ● Self-hosting is now sub-$25

Slide 40

Slide 40 text

Where to go From Here? ● Zero programming experience? ○ Get started with a tutorial about $x ● Looking for something to solve? ○ Pick something annoying ● Can’t figure out how to solve it? ○ Lucky you, you’re at SELF ● Solved something cool? ○ Blog about it and tell us!

Slide 41

Slide 41 text

Thank you! github.com/tylerjl irc/twitter: leothrix tjll.net Additional Information: ● speakerdeck.com/tylerjl ● Corner me anytime this conference ○ (I’ll be at the Elastic booth after)