Tap --[not] a talk about replacing Rake

Slides from my talk at RubyConf 2008

Transcript

1. Tap --[not] a talk about replacing rake Simon Chiang
2. Biomolecular Structure
8. Data Analysis Submit to search engine Pre-process Harvest results ...
9. Design Goals • Each task is independent, configurable • Tasks must have an immediate command line interface • Ability to link tasks together into workflows
10. Originally, Rake desc “a simple goodnight task” task :goodnight, :obj do |task, args| puts “#{ENV[‘msg’] || ‘goodnight’} #{args.obj}” end [Rakefile] % rake -T % rake goodnight[moon] % rake goodnight[world] msg=hello
11. Dependencies x x1 x2 y y1 z
12. The nature of Rake • Insufficient command line support • No imperative workflows • Hard to test/reuse/distribute • Not a knock on rake! • Domain specific solution
13. Tap (Task Application) • A command line framework • Imperative and dependency workflows • Distribution of tasks as gems a b1 b c1 c2 c ? x x1 x2 y y1 z
14. Rake, Rap, Tap
15. Rake Task desc “a simple goodnight task” task :goodnight, :obj do |task, args| puts “#{ENV[‘msg’] || ‘goodnight’} #{args.obj}” end % rake -T % rake goodnight[moon] % rake goodnight[world] msg=hello [Rakefile, rakefile, rakefile.rb]
16. Rap Task # ::desc a simple goodnight task # Extended documentation for the goodnight task... task :goodnight, :obj, :msg => ‘goodnight’ do |task,args| puts “#{task.msg} #{args.obj}” end % rap -T % rap goodnight moon % rap goodnight world --msg hello % rap goodnight --help [Rapfile, rapfile, rapfile.rb]
17. Command Line
18. Quite Compatible • Declarations must be included • No FileList, Rake::TestTask, etc. require ‘tap’ include Tap::Declartions namespace :name do desc ‘one line description’ task({:name => [:deps]}, :inputs, {:key => ‘value’}) do sh “...” end end
19. Rap runs Rake require ‘rake’ desc “says hello” task :say_hello do puts “hello” end require ‘tap’ include Tap::Declarations desc “says goodbye” task :say_goodbye do puts “goodbye” end % rap say_hello % rap say_goodbye % rap -T [Rakefile] [Rapfile]
20. Rap is like a supercharged Rake
21. Class vs Instance • Rake makes instances • Rap makes singleton subclasses t = task :name t.class # => Rake::Task t = task :name t.class # => Name Name.superclass # => Tap::Task Name.instance # => t
22. Beyond Rap # ::desc a simple goodnight task task :goodnight, :obj, :msg => ‘goodnight’ do |task,args| puts “#{task.msg} #{args.obj}” end # ::manifest a simple goodnight task class Goodnight < Tap::Task config :msg, ‘goodnight’ def process(obj) puts “#{msg} #{obj}” end end
23. Anatomy of a Task Class
24. Executable Configurable (Lazydoc)
25. Executable • Wraps a method • Adds workflowability a b1 b c1 c2 c ?
26. Process class Goodnight < Tap::Task def process(obj=‘opus’, *objects) objects.unshift(obj) puts “goodnight #{objects.join(‘,’)}” end end % rap goodnight % rap goodnight moon lamp “little toy boat” % rap goodnight --help
27. Command Line
28. Configurable class Goodnight < Tap::Task config :key, ‘value’ do |input| input.downcase end end t = Goodnight.new t.key # => ‘value’ t.key = ‘New Value’ t.config[:key] # => ‘new value’
29. Standard Blocks class Goodnight < Tap::Task config :message, ‘goodnight’ config :reverse, false, &c.switch config :repeat, 1, &c.integer def process str = “#{message} moon” str = str.reverse if reverse == true repeat.times { puts str } end end % rap goodnight --reverse --repeat 3 % rap goodnight --help
30. Command Line
31. Lazydoc # SomeClass::key this is the value # Extended value... class SomeClass extend Tap::Support::LazyAttributes lazy_attr :key end SomeClass.source_file = __FILE__ SomeClass.key.value # => “this is the value” SomeClass.key.to_s # => “Extended value...”
32. Lazydoc # ::manifest a fancy goodnight task # Says goodnight to lots of objects. class Goodnight < Tap::Task config :message, ‘goodnight’ # a goodnight message config :reverse, false, &c.switch # reverses output config :repeat, 1, &c.integer # repeats output end Goodnight.manifest.value # => “a fancy goodnight task” Goodnight.manifest.to_s # => “Says goodnight to lots of objects.” [lib/goodnight.rb]
33. Command Line
34. Off the command line class Goodnight < Tap::Task config :msg, ‘goodnight’ def process(obj) “#{msg} #{obj}” end end g = Goodnight.new g.process(‘moon’) # => ‘goodnight moon’ g.msg = ‘hello’ g.process(‘world’) # => ‘hello world’
35. Testing class GoodnightTest < Test::Unit::Testcase def test_goodnight_says_goodnight g = Goodnight.new assert_equal ‘goodnight moon’, g.process(‘moon’) end end
36. Subclasses class ReverseGoodnight < Goodnight config :reverse, true, &c.switch def process(obj) super(reverse ? obj.reverse : obj) end end g = ReverseGoodnight.new(:message => ‘hello’) g.process(‘world’) # => ‘dlrow olleh’ g.reverse = false g.process(‘world’) # => ‘hello world’
37. Pass Results g1 = Goodnight.new(:message => ‘yo’) g2 = Goodnight.new(:message => ‘yo’) result = g1.process(‘moon’) g2.process(result) # => ‘yo yo moon’
38. Workflows
39. Executable • Audits execution of the method g = Goodnight.new _result = g._execute(‘moon’) _result.class # => Tap::Support::Audit _result._current # => ‘goodnight moon’ _result._source # => g
40. Executable • Adds an on_complete block current = nil g = Goodnight.new g.on_complete do |_result| current = _result._current end g._execute(‘moon’) current # => ‘goodnight moon’
41. Executable • Allows specification of dependencies dep_ran = false dep = Tap::Task.intern {|task| dep_ran = true } g = Goodnight.new g.depends_on(dep) g._execute dep_ran # => true
42. Build, Enq, Run ExecutableQueue Aggregator Tap::App
43. Example • Simple Trace task, adds name to str class Trace < Tap::Task def process(str) str + name end end t = Trace.new({}, ‘a’) t.name # => ‘a’ t.process(‘’) # => ‘a’ t.process(‘xyz’) # => ‘xyza’
44. Standard Joins a, b, b1, c, c1, c2 = %w{a b b. c c. c..}.map do |name| Tracer.new({}, name) end a.switch(b, b1) do |_result| _result._current == ‘a’ ? 0 : 1 end b.fork(c, c1) c.sequence(a) c2.merge(b1, c1) a b1 b c1 c2 c ?
45. Enque and Run # (could just do this) # a.execute(‘’) app = Tap::App.instance app.enq(a, ‘’) app.run app.results(c2) # => ["abcab.c..", "abc.c.."] a b1 b c1 c2 c ?
46. Audit Trail puts app._results(c2)[0]._to_s # o-[] "" # o-[a] "a" # o-[b] "ab" # o-[c] "abc" # o-[a] "abca" # o-[b.] "abcab." # o-[c..] "abcab.c.." a b1 b c1 c2 c ?
47. Workflow Syntax --- - [task, a, b, c, --key, value] - [task, x, y, z, --key, value] - 0:1 task a b c --key value --: task x y z --key value
48. Delimiters break meaning -- enque next to round 0 --+ enque next to round 1 --++ enque next to round 2 --+3 enque next to round 3 --0:1 sequence 0 -> 1 --0[1,2] fork 0 -> [1,2] --2{0,1} merge [0,1] -> 2 --2(0,1) sync_merge [0,1] -> 2 --* enque a ‘global’ dependency instance
49. Distribution
50. tap: (/Users/simonchiang/Documents/Gems/tap) commands console (cmd/console.rb) destroy (cmd/destroy.rb) generate (cmd/generate.rb) manifest (cmd/manifest.rb) run (cmd/run.rb) generators command (lib/tap/generator/generators/command/command_generator.rb) config (lib/tap/generator/generators/config/config_generator.rb) file_task (lib/tap/generator/generators/file_task/file_task_generator.rb) generator (lib/tap/generator/generators/generator/generator_generator.rb) root (lib/tap/generator/generators/root/root_generator.rb) task (lib/tap/generator/generators/task/task_generator.rb) tasks dump (lib/tap/tasks/dump.rb) load (lib/tap/tasks/load.rb) rake (lib/tap/tasks/rake.rb) Env, Manifests
51. Manifest via Lazydoc module Nested # ::manifest class Sample < Tap::Task end end # Another::manifest class Another < Tap::Task end class Hidden < Tap::Task end [lib/nested/sample.rb] implicit, by path explicitly named overlooked
52. Nested Envs Desktop |- molecules |- sample_tasks `- tap
53. Mini-paths calc molecules:calc molecules-0.1.0:calc.rb /path/to/molecules-0.1.0:/path/to/molecules/lib/calc.rb
54. RubyGems • Package normally (Rapfile doesn’t work) • Add a tap.yml file (can be empty) • Tasks will be discovered automatically
55. Context
56. Server
57. Ubiquity
58. Scripts #!/usr/bin/env ruby require ‘rubygems’ require ‘tap’ # Goodnight::manifest a fancy goodnight task # Says goodnight to lots of objects. class Goodnight < Tap::Task config :message, ‘goodnight’ def process(*objects) puts “#{message} #{objects.join(‘,’)}” end end Goodnight.execute(ARGV)
59. Wrapup • A supercharged rake • A framework for configurable, distributable tasks and workflows
60. Acknowledgements Dr. Kirk Hansen Lauren Kiemele Ashley Zurawel
61. Thanks Mom, Dad, Brother, Girlfriend
62. tap.rubyforge.org