Upgrade to Pro — share decks privately, control downloads, hide ads and more …

How to discover the Ruby's defects with web application

How to discover the Ruby's defects with web application

SHIBATA Hiroshi

December 08, 2012
Tweet

More Decks by SHIBATA Hiroshi

Other Decks in Technology

Transcript

  1. ࣲాതࢤ
    SHIBATA Hiroshi
    QBQFSCPZDP
    BTBLVTBSC
    paperboy&co., Inc.
    Ruby ͷෆ۩߹ͷݟ͚ͭํ
    ൃද৔ॴRubyConf Taiwan 2012 2012-12-8(Sat)
    How to discover the
    Ruby's defects with web
    application.

    View full-size slide

  2. 㟬޷
    ni-hao.

    View full-size slide

  3. จࣈͷς
    ΩετΒ͠
    ͍Ͱ͢Αʁ
    େৎ෉͔ʁ
    Xie Xie rubytaiwan. Lai tai wan, wan hen
    kai xin.

    View full-size slide

  4. SHIBATA Hiroshi(@hsbt)
    Wo shi cong ri ben lai de. Wo jiao Hiroshi,
    Shibata.
    My name is Hiroshi SHIBATA, please call
    me Hiroshi or SHIBATA. This jacket is my
    trademark and fighting armor. Today is
    same jacket.

    View full-size slide

  5. Thanks for @ihower and @rubytaiwan
    accepting my proposal. I appreciate it. I’m
    super exciting.
    Taiwan is good city. I love taiwan. I
    decided to arrived rubyconf taiwan in next
    year.

    View full-size slide

  6. asakusa.rb
    I’m member of asakusa.rb. asakusa.rb is
    inspired by Seattle.rb. This meetup start
    every tuesday. this meetup founder is
    Akira Matsuda. If you arrived tokyo, japan,
    please join our meetup.

    View full-size slide

  7. I’m working at paperboy&co. My job is
    solving for all technical problems such as
    continuous deployment, security issue and
    gaining code quality etc... it’s company
    wide scrum master.

    View full-size slide

  8. http://fanic.jp
    Our one of the rails service is fanic. fanic
    is that you can have personal music store.
    I’m sorry, fanic is only japanese language
    available.

    View full-size slide

  9. http://musiccaptains.com/zh/2012/11/27/fanic-
    music-website-song-purchase/
    You are lucky. Some of chinese writer
    wrote introduction of fanic. this entry is
    chinese!
    Please join it.

    View full-size slide

  10. tDiary
    In this talk, I’ll introduce tDiary and how
    tDiary can discover ruby’s defects. I don’t
    talk agile and how to maintain legacy
    code. If you interested these theme, please
    talk to me after session.

    View full-size slide

  11. http://github.com/tdiary
    tDiary is hosted by github now, It this url.
    I’m sorry this site is only available for
    japanese and english information. if you
    interested tDiary, please join our
    organization.

    View full-size slide

  12. ͆hat’s?
    First, what’s tDiary?
    Do you know tDiary? Please raise up your
    hand?

    View full-size slide

  13. Nikki
    System
    tDiary is Nikki System. Nikki is Diary.
    Japanese internet has web nikki culture
    before blog.

    View full-size slide

  14. Like a Blog
    Nikki is a blog in 2012. We don’t divide
    their.

    View full-size slide

  15. Specification
    Overview
    I’ll introduce tDiary mainly specification.

    View full-size slide

  16. 3. Pluggable
    2. CGI/Rack
    1. Ruby
    First, it use Ruby, and this is most
    important things. Second, running
    environments are traditional CGI and Rack.
    It means tDiary can run in all of
    application server. For example apache,
    passenger and unicorn. Third, it has
    pluggable mechanism. It mechanism is
    possibled by ruby meta-programming.
    this feature supports newbie rubyists.

    View full-size slide

  17. History
    Second, I talk about history.

    View full-size slide

  18. Ruby 1.6.7
    2002
    In 2002, tDiary began to development in
    the sf.net and used CVS. I joined tDiary
    developper team this year.
    First version supported Ruby-1.6.7. Please
    raise up your hands if you had used this
    version of Ruby?

    View full-size slide

  19. Ruby 1.8.0
    2003
    In 2003, tDiary supported Ruby-1.8.0.
    This version included many standard
    libraries.

    View full-size slide

  20. What is Ruby
    needed to run
    the tDiary?
    “What is Ruby needed to run the tDiary.”
    This phrase posted some contents in
    2004. It means tDiary more famous lather
    than Ruby.
    Everyone knows tDiary in Japan.

    View full-size slide

  21. Ruby 1.9.0
    2008
    In 2008, tDiary supported 1.9.0. Big
    internal changing occurred in this version.
    All of String have Encoding. tDiary was
    broken by encoding changes. It was so
    hard for us, and we have be supporting
    now.

    View full-size slide

  22. 2013
    In 2013. it’s future.

    View full-size slide

  23. Ruby
    2.0.0
    Probably, tDiary is supporting 2.0. This
    feature is assigned me.

    View full-size slide

  24. As you can see that tDiary is growing up
    with Ruby.
    In other hand, Ruby is growing up with
    tDiary.

    View full-size slide

  25. tDiary discovered
    Ruby’s defects
    In other words, tDiary discovered some of
    ruby’s defects.

    View full-size slide

  26. why?
    Why tDiary be able to discover Ruby’s
    defects?

    View full-size slide

  27. Background
    I’m going to describe Japanese Internet
    environment. Nearly 2004, web
    programmer need to use shared hosting
    server for web programming.

    View full-size slide

  28. tDiary can use plugin written by tiny ruby
    code. Implementation of this feature need
    to ruby-meta-programming.

    View full-size slide

  29. eval
    tDiary’s core feature is using eval,
    instance_eval and class_eval.

    View full-size slide

  30. private
    # loading tdiary.conf in current directory
    def configure_attrs
    @secure = true unless @secure
    @options = {}
    eval( File::open( 'tdiary.conf' ) {|f| f.read }.untaint, b,
    "(tdiary.conf)", 1 )
    # language setup
    @lang = 'ja' unless @lang
    begin
    This is part of tDiary’s code.
    tDiary configuration was written by Ruby
    code. it’s not by yaml and json.

    View full-size slide

  31. # The value must be the Array of Ruby and Array's contents are
    String
    # of Ruby. Though these Strings are converted to Regular Expression
    # of Ruby when compared, you can't use Regexp of Ruby.
    @no_referer = [
    '^' + Regexp.quote( base_url ), # Your diary
    '^http://localhost[:/]',
    '^http://192.168.',
    '^http://172.1[6789]',
    '^http://172.2[0-9]',
    '^http://172.3[01]',
    '^http://10.',
    ]
    # URLs which are only recorded into Today's Link (Regular Expression)
    # @only_volatile is an array of URLs to record into only Today's
    Link.
    # When a referer match to this list, it will be recoreded to
    volatile
    # list. This list will be cleared when you make new text in new
    day.
    # Specify @only_volatile same style of @no_referer.
    @only_volatile = [
    ]
    # The rules which convert the specified URL to the word (Regular
    Expression)
    # @referer_table is configured so that readable URLs are shown in
    This is example of tdiary configuration
    file.
    If you try to customize tDiary behavior,
    you should write method definitions,
    variable settings and more into
    “tdiary.conf”.
    It’s very useful.

    View full-size slide

  32. private
    def setup_attr_accessor_to_all_ivars
    names = instance_variables().collect {|ivar| ivar.to_s.sub(/
    @/, '') }
    (class << self; self; end).class_eval { attr_accessor
    *names }
    end
    def configure_bot_pattern
    bot = ["bot", "spider", "antenna", "crawler", "moget",
    "slurp"]
    bot += @options['bot'] || []
    @bot = Regexp::new( "(#{bot.uniq.join( '|' )})", true )
    end
    Next point is class_eval.
    tDiary uses classic “cgi.rb” and creates CGI
    instance per all requests.
    CGI instance have a configuration
    instance. It needs access mechanism for
    tDiary’s core and plugins.
    class_eval make configuration instance
    possible to access it.

    View full-size slide

  33. def load_plugin( file )
    @resource_loaded = false
    begin
    res_file = File::dirname( file ) + "/#{@conf.lang}/"
    + File::basename( file )
    open( res_file.untaint ) do |src|
    instance_eval( src.read.untaint, "(plugin/
    #{@conf.lang}/#{File::basename( res_file )})", 1 )
    end
    @resource_loaded = true
    rescue IOError, Errno::ENOENT
    end
    File::open( file.untaint ) do |src|
    instance_eval( src.read.untaint, "(plugin/
    #{File::basename( file )})", 1 )
    end
    end
    The i18n of tDiary use Ruby code. this is
    same the case of configuration files. tDiary
    doesn’t use i18n gem! In past, tDiary
    include chinese language. but we dropped
    it, because we have no maintainer.

    View full-size slide

  34. One of 2.0.0 feature is “Refinements”.
    tDiary have many monkey patches for
    Ruby’s core feature. I’m going to fix these
    patches by using “Refinements”.

    View full-size slide

  35. “eval” is powerful metaprogramming in
    ruby. but you worry about security
    issue.support two things. one, share
    hosting environment. imagine of you
    made cgi in shared hosting environment.
    that cgi can use code written by other
    people. If this code is exploit code. How
    do you protect it?

    View full-size slide

  36. $SAFE
    tDiary protects with variable of $SAFE
    against security issue.
    I’m introducing $SAFE mechanism in Ruby.
    Did you know this feature?

    View full-size slide

  37. open( res_file.untaint ) do |
    src|
    instance_eval( src.read.untaint,
    "(plugin/#{@conf.lang}/
    #{File::basename( res_file )})",
    1 )
    end
    @resource_loaded =
    true
    rescue IOError,
    Errno::ENOENT
    end
    File::open( file.untaint ) do |
    src|
    instance_eval( src.read.untaint,
    "(plugin/
    #{File::basename( file )})", 1 )
    end
    end
    I talk overview of $SAFE mechanism. $SAFE
    mechanism makes all object taint. Once
    you use $SAFE, you can’t use tainted
    object in some function.
    If you want to use tainted objects, you
    should call “untaint” method.

    View full-size slide

  38. $SAFE=1
    $ ruby -e '$SAFE = 1; open(ARGV[0])' foo
    -e:1:in `initialize': Insecure operation -
    initialize (SecurityError)
    from -e:1
    $SAFE level 1 disallow “File.open” with
    tainted strings. If you want to open with
    ARGV[0] string. you should call “untaint”.

    View full-size slide

  39. $SAFE=4
    % ruby -e '$SAFE=4; eval("p :foo".untaint)'
    -e:1:in `untaint': Insecure operation at level 4 (SecurityError)
    from -e:1:in `'
    % ruby -e '$SAFE=1; eval("p :foo".untaint)'
    :foo
    $SAFE level 4. If you use untainted strings
    with eval, this level disallow using it.
    tDiary customization is handled with eval
    and $SAFE mechanism.

    View full-size slide

  40. =begin
    == Safe module
    =end
    module Safe
    def safe( level = 4 )
    result = nil
    if $SAFE < level then
    Proc.new {
    $SAFE = level
    result = yield
    }.call
    else
    result = yield
    end
    result
    end
    module_function :safe
    end
    It’s protips. If you use different $SAFE level
    in same code. you need to use “Proc#new”
    and “Proc#call”.
    tDiary use this procedure when running
    with shared environment.

    View full-size slide

  41. bugreport
    http://bugs.ruby-lang.org/issues/5279
    fixed r33328:
    * encoding.c (require_enc): reject only loading from untrusted
    load paths. [ruby-dev:44541] [Bug #5279]
    * transcode.c (load_transcoder_entry): ditto. assume system
    default tmpdir safe. [ruby-dev:42089]
    http://bugs.ruby-lang.org/issues/3733
    fixed r29209: assume system default tmpdir safe. [ruby-dev:42089]
    http://bugs.ruby-lang.org/issues/1115
    * load.c (rb_require_safe): raises when the path to be loaded is
    tainted. [ruby-dev:37843]
    I've never seen that other people are using
    $SAFE feature. therefore $SAFE feature is
    very buggy.

    View full-size slide

  42. http://bugs.ruby-lang.org/issues/1115
    * load.c (rb_require_safe): raises when the path to be loaded is tainted. [ruby-dev:37843]
    % ruby -e '$SAFE=1;p require "zlib"'
    -e:1:in `require': Insecure operation -
    require (SecurityError)
    from -e:1:in `'
    % ruby -e '$SAFE=1;p require "English"'
    -e:1:in `require': Insecure operation -
    require (SecurityError)
    from -e:1:in `'
    This defect is that ruby raises exception
    when ruby call require with tainted path.

    View full-size slide

  43. http://bugs.ruby-lang.org/issues/3733
    fixed r29209: assume system default tmpdir safe. [ruby-dev:42089]
    % ruby -e
    '$SAFE=1;File.expand_path(".".taint)'
    -e:1:in `expand_path': Insecure operation -
    expand_path (SecurityError)
    from -e:1:in `'
    This defects. if you use “tmpdir” running
    with $SAFE level 4, Ruby raised exception.

    View full-size slide

  44. http://bugs.ruby-lang.org/issues/5279
    fixed r33328:
    * encoding.c (require_enc): reject only loading from untrusted
    load paths. [ruby-dev:44541] [Bug #5279]
    * transcode.c (load_transcoder_entry): ditto. assume system default tmpdir safe.
    [ruby-dev:42089]
    % ruby -e ‘$SAFE = 3; "a".encode("UTF-16")’
    -e:1:in `encode': Insecure operation - encode
    (SecurityError)
    from -e:1:in `'
    In this report. If you run some code with
    $SAFE level 4, and not with calling
    “encode”. Ruby is raised exception.
    I already talked about all of String has
    Encoding in Ruby 1.9.
    This defects is important, because anyone
    never use String#encode with $SAFE level
    4 in Ruby 1.9

    View full-size slide

  45. จࣈͷς
    ΩετΒ͠
    ͍Ͱ͢Αʁ
    େৎ෉͔ʁ
    I’m sorry, I introduce bad news. JRuby isn’t
    support $SAFE yet.

    View full-size slide

  46. @shugomaeda
    4"'&VTFSTJTPOMZ
    U%JBSZ*XBOUUPESPQ
    JUJOOFYUWFSTJPOPG
    SVCZ
    SFBMMZ
    @hsbt
    I think probably dropped out $SAFE
    feature in 2.1.
    We accept it. so tDiary need to change
    with rubies. The Internet environment is
    changing now. people doesn’t use shared
    hosting server. they use PaaS like a sqale
    and heroku.

    View full-size slide

  47. tDiary discover ruby’s defects is easy. In
    this case is only tDiary being?
    I think no.

    View full-size slide

  48. tDiary uses Travis-CI.
    I build ruby-trunk every morning, and run
    it with tDiary. So, sometimes I get defects
    in ruby-trunk.

    View full-size slide

  49. @tenderlove said that Rails-3.2 works with
    Ruby-2.0.
    It means that many web applications can
    work with Ruby-2.0. You can begin to use
    Ruby-2.0 now.

    View full-size slide

  50. Even though, you couldn’t run code with
    Ruby-2.0 in production, you can run code
    in test environment.
    I think modern code have a lot of test
    code. you can.

    View full-size slide

  51. https://speakerdeck.com/kakutani/above-all-make-it-fun
    If it was also difficult to run your code in
    test environment, your hobby code should
    run with Ruby-2.0.
    I think that programmer have hobby code.
    tDiary is hobby code. Matz said “Ruby is
    hobby code!” too.

    View full-size slide

  52. FFFFFFFFFFFFFFFFFFF.....F
    FFFFFFFFFFFFF....F.F...FFFF
    FFFFFFFFFFFFFFFFFFFFFF.
    ....FFFFFFFFFFFFFF....F.F...F
    FFFFFFFFFFFFFFFFFFFFFF
    FFF.....FFFFFFFFFFFFFF....F.
    F...FFFFFFF.......FFFFFFFFFF
    When you run hobby code with Ruby-2.0,
    you will get error or segmentation fault.
    You should report it ruby core team.

    View full-size slide

  53. bugreport
    http://bugs.ruby-lang.org/issues/6781
    * lib/open-uri.rb: use respond_to? to test Tempfile.
    [ruby-dev:45995] [Bug #6781] reported by hsbt (Hiroshi SHIBATA).
    http://bugs.ruby-lang.org/issues/5952
    * io.c (argf_next_argv): reset ARGF.next_p on ARGV.replace.
    r34409 breaks replacing ARGV.
    [ruby-dev:45160] [Bug #5952]
    http://bugs.ruby-lang.org/issues/7040
    * ext/zlib/zlib.c (zstream_run_func): don't call inflate() when
    z->stream.avail_in == 0. it return Z_BUF_ERROR.
    but deflate() could be called with z->stream->avail_in == 0 because
    it has hidden buffer in z->stream->state (opaque structure).
    fix for gem install error. [ruby-dev:46149] [Bug #7040]
    I discovered some defects in ruby-trunk.
    One of these defects is ruby-trunk can’t
    install some gems like libv8. Today’s ruby
    is fixed by my reports.

    View full-size slide

  54. A few weeks ago. I discovered a new
    feature in cgi.rb. I thought this feature
    broke backward compatibility. I reported it
    for ruby-core.
    In this result, this feature reverted.
    You need to run your code with early
    version of Ruby. and report defects,
    behavior change and new feature.
    If you detect behavior change after code
    freeze, it’s too late.

    View full-size slide

  55. Report
    it!
    You shouldn’t write these report in blog or
    twitter. Probably, ruby-core commiters
    don’t read your tweets.

    View full-size slide

  56. Are commiter only people who develop
    ruby? I don’t think so. this picture is one
    of scene in rubyconf. matz, ko1 and JRuby
    guys and many commiter and people
    discussion “refinements”
    Your report and proposal of new feature
    are making ruby!

    View full-size slide

  57. Run your code
    with 2.0.0
    Let’s run your code with ruby 2.0.0

    View full-size slide