Upgrade to Pro — share decks privately, control downloads, hide ads and more …

How to discover the Ruby's defects with web application

How to discover the Ruby's defects with web application


December 08, 2012

More Decks by SHIBATA Hiroshi

Other Decks in Technology


  1. ࣲాതࢤ SHIBATA Hiroshi QBQFSCPZDP BTBLVTBSC paperboy&co., Inc. Ruby ͷෆ۩߹ͷݟ͚ͭํ ൃද৔ॴRubyConf

    Taiwan 2012 2012-12-8(Sat) How to discover the Ruby's defects with web application.
  2. SHIBATA Hiroshi(@hsbt) Wo shi cong ri ben lai de. Wo

    jiao Hiroshi, Shibata. My name is Hiroshi SHIBATA, please call me Hiroshi or SHIBATA. This jacket is my trademark and fighting armor. Today is same jacket.
  3. Thanks for @ihower and @rubytaiwan accepting my proposal. I appreciate

    it. I’m super exciting. Taiwan is good city. I love taiwan. I decided to arrived rubyconf taiwan in next year.
  4. asakusa.rb I’m member of asakusa.rb. asakusa.rb is inspired by Seattle.rb.

    This meetup start every tuesday. this meetup founder is Akira Matsuda. If you arrived tokyo, japan, please join our meetup.
  5. I’m working at paperboy&co. My job is solving for all

    technical problems such as continuous deployment, security issue and gaining code quality etc... it’s company wide scrum master.
  6. http://fanic.jp Our one of the rails service is fanic. fanic

    is that you can have personal music store. I’m sorry, fanic is only japanese language available.
  7. tDiary In this talk, I’ll introduce tDiary and how tDiary

    can discover ruby’s defects. I don’t talk agile and how to maintain legacy code. If you interested these theme, please talk to me after session.
  8. http://github.com/tdiary tDiary is hosted by github now, It this url.

    I’m sorry this site is only available for japanese and english information. if you interested tDiary, please join our organization.
  9. Nikki System tDiary is Nikki System. Nikki is Diary. Japanese

    internet has web nikki culture before blog.
  10. 3. Pluggable 2. CGI/Rack 1. Ruby First, it use Ruby,

    and this is most important things. Second, running environments are traditional CGI and Rack. It means tDiary can run in all of application server. For example apache, passenger and unicorn. Third, it has pluggable mechanism. It mechanism is possibled by ruby meta-programming. this feature supports newbie rubyists.
  11. Ruby 1.6.7 2002 In 2002, tDiary began to development in

    the sf.net and used CVS. I joined tDiary developper team this year. First version supported Ruby-1.6.7. Please raise up your hands if you had used this version of Ruby?
  12. What is Ruby needed to run the tDiary? “What is

    Ruby needed to run the tDiary.” This phrase posted some contents in 2004. It means tDiary more famous lather than Ruby. Everyone knows tDiary in Japan.
  13. Ruby 1.9.0 2008 In 2008, tDiary supported 1.9.0. Big internal

    changing occurred in this version. All of String have Encoding. tDiary was broken by encoding changes. It was so hard for us, and we have be supporting now.
  14. As you can see that tDiary is growing up with

    Ruby. In other hand, Ruby is growing up with tDiary.
  15. Background I’m going to describe Japanese Internet environment. Nearly 2004,

    web programmer need to use shared hosting server for web programming.
  16. tDiary can use plugin written by tiny ruby code. Implementation

    of this feature need to ruby-meta-programming.
  17. private # loading tdiary.conf in current directory def configure_attrs @secure

    = true unless @secure @options = {} eval( File::open( 'tdiary.conf' ) {|f| f.read }.untaint, b, "(tdiary.conf)", 1 ) # language setup @lang = 'ja' unless @lang begin This is part of tDiary’s code. tDiary configuration was written by Ruby code. it’s not by yaml and json.
  18. # The value must be the Array of Ruby and

    Array's contents are String # of Ruby. Though these Strings are converted to Regular Expression # of Ruby when compared, you can't use Regexp of Ruby. @no_referer = [ '^' + Regexp.quote( base_url ), # Your diary '^http://localhost[:/]', '^http://192.168.', '^http://172.1[6789]', '^http://172.2[0-9]', '^http://172.3[01]', '^http://10.', ] # URLs which are only recorded into Today's Link (Regular Expression) # @only_volatile is an array of URLs to record into only Today's Link. # When a referer match to this list, it will be recoreded to volatile # list. This list will be cleared when you make new text in new day. # Specify @only_volatile same style of @no_referer. @only_volatile = [ ] # The rules which convert the specified URL to the word (Regular Expression) # @referer_table is configured so that readable URLs are shown in This is example of tdiary configuration file. If you try to customize tDiary behavior, you should write method definitions, variable settings and more into “tdiary.conf”. It’s very useful.
  19. private def setup_attr_accessor_to_all_ivars names = instance_variables().collect {|ivar| ivar.to_s.sub(/ @/, '')

    } (class << self; self; end).class_eval { attr_accessor *names } end def configure_bot_pattern bot = ["bot", "spider", "antenna", "crawler", "moget", "slurp"] bot += @options['bot'] || [] @bot = Regexp::new( "(#{bot.uniq.join( '|' )})", true ) end Next point is class_eval. tDiary uses classic “cgi.rb” and creates CGI instance per all requests. CGI instance have a configuration instance. It needs access mechanism for tDiary’s core and plugins. class_eval make configuration instance possible to access it.
  20. def load_plugin( file ) @resource_loaded = false begin res_file =

    File::dirname( file ) + "/#{@conf.lang}/" + File::basename( file ) open( res_file.untaint ) do |src| instance_eval( src.read.untaint, "(plugin/ #{@conf.lang}/#{File::basename( res_file )})", 1 ) end @resource_loaded = true rescue IOError, Errno::ENOENT end File::open( file.untaint ) do |src| instance_eval( src.read.untaint, "(plugin/ #{File::basename( file )})", 1 ) end end The i18n of tDiary use Ruby code. this is same the case of configuration files. tDiary doesn’t use i18n gem! In past, tDiary include chinese language. but we dropped it, because we have no maintainer.
  21. One of 2.0.0 feature is “Refinements”. tDiary have many monkey

    patches for Ruby’s core feature. I’m going to fix these patches by using “Refinements”.
  22. “eval” is powerful metaprogramming in ruby. but you worry about

    security issue.support two things. one, share hosting environment. imagine of you made cgi in shared hosting environment. that cgi can use code written by other people. If this code is exploit code. How do you protect it?
  23. $SAFE tDiary protects with variable of $SAFE against security issue.

    I’m introducing $SAFE mechanism in Ruby. Did you know this feature?
  24. open( res_file.untaint ) do | src| instance_eval( src.read.untaint, "(plugin/#{@conf.lang}/ #{File::basename(

    res_file )})", 1 ) end @resource_loaded = true rescue IOError, Errno::ENOENT end File::open( file.untaint ) do | src| instance_eval( src.read.untaint, "(plugin/ #{File::basename( file )})", 1 ) end end I talk overview of $SAFE mechanism. $SAFE mechanism makes all object taint. Once you use $SAFE, you can’t use tainted object in some function. If you want to use tainted objects, you should call “untaint” method.
  25. $SAFE=1 $ ruby -e '$SAFE = 1; open(ARGV[0])' foo -e:1:in

    `initialize': Insecure operation - initialize (SecurityError) from -e:1 $SAFE level 1 disallow “File.open” with tainted strings. If you want to open with ARGV[0] string. you should call “untaint”.
  26. $SAFE=4 % ruby -e '$SAFE=4; eval("p :foo".untaint)' -e:1:in `untaint': Insecure

    operation at level 4 (SecurityError) from -e:1:in `<main>' % ruby -e '$SAFE=1; eval("p :foo".untaint)' :foo $SAFE level 4. If you use untainted strings with eval, this level disallow using it. tDiary customization is handled with eval and $SAFE mechanism.
  27. =begin == Safe module =end module Safe def safe( level

    = 4 ) result = nil if $SAFE < level then Proc.new { $SAFE = level result = yield }.call else result = yield end result end module_function :safe end It’s protips. If you use different $SAFE level in same code. you need to use “Proc#new” and “Proc#call”. tDiary use this procedure when running with shared environment.
  28. bugreport http://bugs.ruby-lang.org/issues/5279 fixed r33328: * encoding.c (require_enc): reject only loading

    from untrusted load paths. [ruby-dev:44541] [Bug #5279] * transcode.c (load_transcoder_entry): ditto. assume system default tmpdir safe. [ruby-dev:42089] http://bugs.ruby-lang.org/issues/3733 fixed r29209: assume system default tmpdir safe. [ruby-dev:42089] http://bugs.ruby-lang.org/issues/1115 * load.c (rb_require_safe): raises when the path to be loaded is tainted. [ruby-dev:37843] I've never seen that other people are using $SAFE feature. therefore $SAFE feature is very buggy.
  29. http://bugs.ruby-lang.org/issues/1115 * load.c (rb_require_safe): raises when the path to be

    loaded is tainted. [ruby-dev:37843] % ruby -e '$SAFE=1;p require "zlib"' -e:1:in `require': Insecure operation - require (SecurityError) from -e:1:in `<main>' % ruby -e '$SAFE=1;p require "English"' -e:1:in `require': Insecure operation - require (SecurityError) from -e:1:in `<main>' This defect is that ruby raises exception when ruby call require with tainted path.
  30. http://bugs.ruby-lang.org/issues/3733 fixed r29209: assume system default tmpdir safe. [ruby-dev:42089] %

    ruby -e '$SAFE=1;File.expand_path(".".taint)' -e:1:in `expand_path': Insecure operation - expand_path (SecurityError) from -e:1:in `<main>' This defects. if you use “tmpdir” running with $SAFE level 4, Ruby raised exception.
  31. http://bugs.ruby-lang.org/issues/5279 fixed r33328: * encoding.c (require_enc): reject only loading from

    untrusted load paths. [ruby-dev:44541] [Bug #5279] * transcode.c (load_transcoder_entry): ditto. assume system default tmpdir safe. [ruby-dev:42089] % ruby -e ‘$SAFE = 3; "a".encode("UTF-16")’ -e:1:in `encode': Insecure operation - encode (SecurityError) from -e:1:in `<main>' In this report. If you run some code with $SAFE level 4, and not with calling “encode”. Ruby is raised exception. I already talked about all of String has Encoding in Ruby 1.9. This defects is important, because anyone never use String#encode with $SAFE level 4 in Ruby 1.9

    dropped out $SAFE feature in 2.1. We accept it. so tDiary need to change with rubies. The Internet environment is changing now. people doesn’t use shared hosting server. they use PaaS like a sqale and heroku.
  33. tDiary uses Travis-CI. I build ruby-trunk every morning, and run

    it with tDiary. So, sometimes I get defects in ruby-trunk.
  34. @tenderlove said that Rails-3.2 works with Ruby-2.0. It means that

    many web applications can work with Ruby-2.0. You can begin to use Ruby-2.0 now.
  35. Even though, you couldn’t run code with Ruby-2.0 in production,

    you can run code in test environment. I think modern code have a lot of test code. you can.
  36. https://speakerdeck.com/kakutani/above-all-make-it-fun If it was also difficult to run your code

    in test environment, your hobby code should run with Ruby-2.0. I think that programmer have hobby code. tDiary is hobby code. Matz said “Ruby is hobby code!” too.
  37. bugreport http://bugs.ruby-lang.org/issues/6781 * lib/open-uri.rb: use respond_to? to test Tempfile. [ruby-dev:45995]

    [Bug #6781] reported by hsbt (Hiroshi SHIBATA). http://bugs.ruby-lang.org/issues/5952 * io.c (argf_next_argv): reset ARGF.next_p on ARGV.replace. r34409 breaks replacing ARGV. [ruby-dev:45160] [Bug #5952] http://bugs.ruby-lang.org/issues/7040 * ext/zlib/zlib.c (zstream_run_func): don't call inflate() when z->stream.avail_in == 0. it return Z_BUF_ERROR. but deflate() could be called with z->stream->avail_in == 0 because it has hidden buffer in z->stream->state (opaque structure). fix for gem install error. [ruby-dev:46149] [Bug #7040] I discovered some defects in ruby-trunk. One of these defects is ruby-trunk can’t install some gems like libv8. Today’s ruby is fixed by my reports.
  38. A few weeks ago. I discovered a new feature in

    cgi.rb. I thought this feature broke backward compatibility. I reported it for ruby-core. In this result, this feature reverted. You need to run your code with early version of Ruby. and report defects, behavior change and new feature. If you detect behavior change after code freeze, it’s too late.
  39. Report it! You shouldn’t write these report in blog or

    twitter. Probably, ruby-core commiters don’t read your tweets.
  40. Are commiter only people who develop ruby? I don’t think

    so. this picture is one of scene in rubyconf. matz, ko1 and JRuby guys and many commiter and people discussion “refinements” Your report and proposal of new feature are making ruby!