James Edward Gray II ‣I run the Ruby Quiz ‣I wrote some open source libraries ‣FasterCSV ‣HighLine ‣I authored a couple of Pragmatic books with Ruby in them
James Edward Gray II ‣I run the Ruby Quiz ‣I wrote some open source libraries ‣FasterCSV ‣HighLine ‣I authored a couple of Pragmatic books with Ruby in them ‣I maintain the Ruby bundle for TextMate
Glue Languages ‣A design goal of Perl was to make it a good “glue language” ‣Glue languages are used to join a set of external tools together to get work done
Glue Languages ‣A design goal of Perl was to make it a good “glue language” ‣Glue languages are used to join a set of external tools together to get work done ‣Ruby copied this Perlism
Evil Experts ‣Multiple books warn programmers away from glue features ‣Experts claim ‣Using these features hurts portability ‣Using these features adds failure points
I Have a Super Power ‣I’m immune to the word “can’t” ‣We, as an industry, sometimes struggle with that word ‣MJD once said: Programming is a young field and when alchemy was as young as we are now, they were still trying to turn lead into gold
We May not Need/ Want Portability ‣If we know where the code will run, there’s no problem ‣TextMate uses Mac OS X glue code ‣Rails applications deployed to a company server have a known platform
We May not Need/ Want Portability ‣If we know where the code will run, there’s no problem ‣TextMate uses Mac OS X glue code ‣Rails applications deployed to a company server have a known platform ‣We may be accessing platform specific features like AppleScript, Spotlight, or Plist API’s
Libraries Fail Too ‣C extensions can have non-trivial or non-portable installs ‣Dependencies make this even worse ‣Libraries throw errors you must handle as well
Counter Argument: It’s Fast! ‣At work, I investigated options for an HTML to PDF conversion job ‣The Good Way: PDF::Writer ‣The Evil Way: wrap `html2ps | ps2pdf`
Counter Argument: It’s Fast! ‣At work, I investigated options for an HTML to PDF conversion job ‣The Good Way: PDF::Writer ‣The Evil Way: wrap `html2ps | ps2pdf` ‣I gave each approach three hours of my time ‣I estimated PDF::Writer would take weeks ‣I basically finished the job with glue code
Example: A Unique ID ‣A common need ‣Asked a lot on Ruby Talk ‣The last thread included ideas from a lot of smart people ‣There are multiple Libraries for this
Alternate Syntax ‣Use this syntax when you need backticks in your command ‣any symbol can be a delimiter ‣You can also use the matching pairs: (…), […], {…}, and <…> ‣These nest properly id = %x{uuidgen} id = %[email protected]@
Example: The Pasteboard ‣I want to put a search string on OS X’s find “pasteboard” (clipboard) ‣I don’t need any output for this operation ‣I just need to know if the operation succeeded ‣A simple true or false will do
Shell Expansion ENV["MY_VAR"] = "Set from Ruby" ! system "echo $MY_VAR" # >> Set from Ruby ! system "echo", "$MY_VAR" # >> $MY_VAR ‣A single argument goes through shell expansion ‣File glob patterns ‣Environment variables
Shell Expansion ENV["MY_VAR"] = "Set from Ruby" ! system "echo $MY_VAR" # >> Set from Ruby ! system "echo", "$MY_VAR" # >> $MY_VAR ‣A single argument goes through shell expansion ‣File glob patterns ‣Environment variables ‣Multiple arguments are passed without going through expansion
Example: Backups ‣I want to backup a directory as part of a larger automation ‣The rsync program can do what I need ‣I need to watch for problems and handle them gracefully ‣Possibly emailing a warning to the user
Proper Shell Escaping # escape text to make it useable in a shell script as # one “word” (string) def escape_for_shell(str) str.to_s.gsub( /(?=[^a-zA-Z0-9_.\/\-\x7F-\xFF\n])/, '\\' ). gsub( /\n/, "'\n'" ). sub( /^$/, "''" ) end
Tips for Avoiding Errors ‣Use full paths to programs and files whenever possible ‣Send data to STDIN when you can ‣If you can’t send it to STDIN, dump the data to a Tempfile and send that path
Tips for Avoiding Errors ‣Use full paths to programs and files whenever possible ‣Send data to STDIN when you can ‣If you can’t send it to STDIN, dump the data to a Tempfile and send that path ‣Remember to shell escape any command- line arguments that could contain dangerous characters (even spaces)
Example: Formatting Prose ‣I want to rewrap some prose provided by the user ‣Command-line arguments are not appropriate here ‣Complex shell Escaping ‣Size limit
Example: Formatting Prose ‣I want to rewrap some prose provided by the user ‣Command-line arguments are not appropriate here ‣Complex shell Escaping ‣Size limit ‣I need to send the prose to fmt via STDIN
Reading and Writing prose = <Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum. END_PROSE ! formatted = IO.popen("fmt -w 30", "r+") do |pipe| # open("| fmt -w 30", "r+") do |pipe| pipe << prose pipe.close_write pipe.read end
Reading and Writing prose = <Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum. END_PROSE ! formatted = IO.popen("fmt -w 30", "r+") do |pipe| # open("| fmt -w 30", "r+") do |pipe| pipe << prose pipe.close_write pipe.read end
Reading and Writing prose = <Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum. END_PROSE ! formatted = IO.popen("fmt -w 30", "r+") do |pipe| # open("| fmt -w 30", "r+") do |pipe| pipe << prose pipe.close_write pipe.read end
Reading and Writing prose = <Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum. END_PROSE ! formatted = IO.popen("fmt -w 30", "r+") do |pipe| # open("| fmt -w 30", "r+") do |pipe| pipe << prose pipe.close_write pipe.read end
Reading and Writing prose = <Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum. END_PROSE ! formatted = IO.popen("fmt -w 30", "r+") do |pipe| # open("| fmt -w 30", "r+") do |pipe| pipe << prose pipe.close_write pipe.read end
Example: A Ruby Session ‣I want to run some Ruby code ‣I don’t want that code to affect my current Ruby process ‣I may also need to do some special setup, hacking Ruby’s core, before this code is run
Example: A Ruby Session ‣I want to run some Ruby code ‣I don’t want that code to affect my current Ruby process ‣I may also need to do some special setup, hacking Ruby’s core, before this code is run ‣I need to format STDOUT and STDERR differently
With Error Handling require "open3" ! Open3.popen3("ruby") do |stdin, stdout, stderr| stdin << %Q{puts "I am a puppet."; oops!()} stdin.close puts "Output:" puts stdout.read puts "Errors:" puts stderr.read end
With Error Handling require "open3" ! Open3.popen3("ruby") do |stdin, stdout, stderr| stdin << %Q{puts "I am a puppet."; oops!()} stdin.close puts "Output:" puts stdout.read puts "Errors:" puts stderr.read end
With Error Handling require "open3" ! Open3.popen3("ruby") do |stdin, stdout, stderr| stdin << %Q{puts "I am a puppet."; oops!()} stdin.close puts "Output:" puts stdout.read puts "Errors:" puts stderr.read end
With Error Handling require "open3" ! Open3.popen3("ruby") do |stdin, stdout, stderr| stdin << %Q{puts "I am a puppet."; oops!()} stdin.close puts "Output:" puts stdout.read puts "Errors:" puts stderr.read end
When you Also Need a PID ‣Install the POpen4 gem ‣Unix version ‣Windows versions ‣popen4() works like popen3() but it also passes you the PID for the child process ‣The PID is useful for sending the child process signals, possibly to kill the process
Example: Tracking Ruby ‣I want to download the latest version of Ruby as part of a larger automation ‣I want to verify the contents of the download ‣I want to expand the compressed archive
Use Caution ‣These scraping techniques see wider use than talking to external processes ‣Ironically, they really do seem to be more fragile ‣tips for managing scraping code: ‣Abstract out the scraping code ‣Use more aggressive error handling ‣Make sure the maintenance is worth it
jeg2
kikuchis
eiki
salaboy
niryuu
washizaki
_kensh
shownet
isaoshimizu
naoto_pq
satoshirobatofujimoto
bnasslahsen
honma12345
jrom
speakerdeck
morganepeng
aleyda
marcelosomers
ddemaree
jeffersonlam
marktimemedia
sferik
edds
samlambert
addyosmani