Why Rails is Hard

Why Rails is Hard

428167a3ec72235ba971162924492609?s=128

Yehuda Katz

April 30, 2012
Tweet

Transcript

  1. There are known unknowns... WHY RAILS IS HARD

  2. ...and unknown unknowns WHY OPEN SOURCE IS HARD

  3. [T]here are known knowns; there are things we know we

    know. We also know there are known unknowns; that is to say we know there are some things we do not know. But there are also unknown unknowns – there are things we do not know we don't know. “ DONALD RUMSFELD
  4. OPEN SOURCE. unknown unknowns known knowns known unknowns

  5. OPEN SOURCE. normal rails developer

  6. OPEN SOURCE. sinatra or node developer

  7. OPEN SOURCE. framework developer

  8. KNOWN KNOWNS. web browser web server router app

  9. KNOWN KNOWNS. web browser passenger / unicorn rails router rails

    action
  10. KNOWN KNOWNS. web browser node server node router node app

  11. EQUIVALENT? web browser passenger / unicorn rails router rails action

    web browser node server node router node app =
  12. KNOWN UNKNOWNS

  13. CSRF PROTECTION

  14. CONNECT.

  15. This middleware requires session support, thus should be added somewhere

    below session() and cookieParser(). “ CONNECT DOCS
  16. By default this middleware generates a token named "_csrf" which

    should be added to requests which mutate state, within a hidden form eld, query-string etc. This token is validated against the visitor's req.session._csrf property. “ CONNECT DOCS
  17. NEVER ENLIST END DEVELOPERS IN FRAMEWORK SECURITY.

  18. class CashTransferController def create authorize! current_user, params transfer = CashTransfer.new(

    params[:from_account], params[:to_account], params[:amount] ) queue.push transfer end end WHAT IS CSRF?
  19. class CashTransferController def create authorize! current_user, params transfer = CashTransfer.new(

    params[:from_account], params[:to_account], params[:amount] ) queue.push transfer end end WHAT IS CSRF?
  20. rails browser POST /login username: wycats password: <3<3<3<3<3

  21. rails browser Set-Cookie: userid=5a73678b7b674

  22. <html> <h1>Send Money!</h1> <form action='bank.com/transfer'> <input type='text' name='from' value='me'> <input

    type='text' name='to' value='you'> <input type='text' name='amount' value='1biiiiiilliondollars'> <button type="submit">Transfer</button> </form> </html> BANK.COM
  23. rails browser POST /transfer from=me&to=you&amount=1bi iiiiilliondollars Cookie: userid=5a73678b7b674 automatic POST

    from bank.com
  24. var form = "<form action='bank.com/transfer'>" + "<input type='text' name='from' value='me'>"

    + "<input type='text' name='to' value='you'>" + "<input type='text' name='amount' value='1biiiiiilliondollars'>" + "<button type="submit">hehehehe</button>" + "</form>"; $(form).appendTo("body").submit(); EVIL.COM.
  25. rails browser POST /transfer from=me&to=you&amount=1bi iiiiilliondollars Cookie: userid=5a73678b7b674 automatic POST

    from evil.com
  26. <html> <h1>Send Money!</h1> <form action='bank.com/transfer'> <input type='hidden' name='csrf_token' value='b674005056434a48054707d'> <input

    type='text' name='from' value='me'> <input type='text' name='to' value='you'> <input type='text' name='amount' value='1biiiiiilliondollars'> <button type="submit">Transfer</button> </form> </html> MITIGATION.
  27. THE GOAL IS DIFFERENTIATING YOUR POST FROM THIRD-PARTY POST.

  28. ▪ On by default ▪ Works for all common use-cases

    ▪ HTML forms ▪ Ajax requests ▪ API clients ▪ Not vulnerable to known attacks ▪ Earlier "known good" approaches turned out vulnerable to Flash exploit FRAMEWORK GOALS.
  29. ON BY DEFAULT.

  30. AVOID DISABLING.

  31. WORKS FOR ALL COMMON USE CASES.

  32. FORMS.

  33. 100% MANUAL.

  34. <form action="/cash_transfer"> {% csrf_token %} SEMIAUTOMATIC.

  35. <%= form_for @cash_transfer do |f| %> <%= f.text_field :from %>

    <%= f.text_field :to %> <%= f.text_field :amount %> <%= button "Transfer!" %> <% end %> AUTOMATIC.
  36. HTTP METHODS.

  37. HEAD SAFE GET SAFE POST UNSAFE PUT UNSAFE DELETE UNSAFE

  38. HEAD UNSAFE GET UNSAFE POST UNSAFE PUT UNSAFE DELETE UNSAFE

    Connect Middleware
  39. HEAD SAFE GET SAFE POST UNSAFE PUT SAFE DELETE SAFE

    Connect Docs
  40. POST-ONLY!

  41. AJAX REQUESTS. UNSAFE

  42. RACK-PROTECTION.

  43. AJAX REQUESTS. SAFE

  44. SEMIAUTOMATIC.

  45. AVOID DISABLING.

  46. AVOID DISABLING. I've searched around SO and saw some information

    about turning off the CSRF check for my view via the csrf_exempt decorator, but I nd that unappealing
  47. Generated index.html.erb: <head> <%= csrf_meta_tags %> <%= javascript_include_tag "application" %>

    </head> Included rails.js: $.ajaxPrefilter(function(options, originalOptions, xhr) { if ( !options.crossDomain ) { rails.CSRFProtection(xhr); } }); rails.CSRFProtection = function(xhr) { var token = $('meta[name="csrf-token"]').attr('content'); if (token) { xhr.setRequestHeader('X-CSRF-Token', token); } }; RAILS.
  48. UNKNOWN UNKNOWNS.

  49. ENCODINGS.

  50. "Yehüda".encode("ISO-8859-1").dump => "Yeh\xFCda" "Yehüda".encode("UTF-8").dump => "Yeh\xC3xBCda" "Yehüda".encode("UTF-16").dump => "\xFE\xFF\x00Y\x00e\x00h\x00\xFC\x00d \x00a"

    STRINGS.
  51. "Yehüda".encode("ISO-8859-1") => "\x59\x65\x68\xFC\x64\x61" "Yehüda".encode("UTF-8") => "\x59\x65\x68\xC3xBC\x64\x61" "Yehüda".encode("UTF-16") => "\xFE\xFF\x00Y\x00e\x00h\x00\xFC\x00d \x00a"

    STRINGS.
  52. # share 7 bits of ASCII "Yehuda".encode("UTF-8").bytes.to_a == "Yehuda".encode("ISO-8859-1").bytes.to_a =>

    true # differ on the remaining bit "Yehüda".encode("UTF-8").bytes.to_a == "Yehüda".encode("ISO-8859-1").bytes.to_a => false UTF-8 VS. LATIN-1.
  53. # What does this mean? "\x59\x65\x68\xFC\x64\x61" # Without more metadata,

    unknown BINARY DATA.
  54. # What does this mean? "\x59\x65\x68\xFC\x64\x61" # Without more metadata,

    unknown # ... we can try to guess ... BINARY DATA.
  55. "Yehüda" (latin 1) "Yeh\xFCda" + "Yehüda" (UTF-8) "Yeh\xC3xBCda" = "Yeh\xFCdaYeh\xC3xBCda"

    MIXING.
  56. "Yeh\xFCdaYeh\xC3xBCda" (as Latin-1): YehüdaYehüda INTERPRETING.

  57. "Yeh\xFCdaYeh\xC3xBCda" (as Latin-1): YehüdaYehüda "Yeh\xFCdaYeh\xC3xBCda" (as UTF-8): Yeh daYehüda INTERPRETING.

  58. A WILD REPLACEMENT CHARACTER APPEARS

  59. HTTP 1.1/200 OK Content-Type: application/json; charset=UTF-8 Yehüda Katz "\x59\x65\x68\xC3xBC\x64\x61" FIGURING

    IT OUT.
  60. HTTP 1.1/200 OK Content-Type: application/json; charset=UTF-8 Yehüda Katz "\x59\x65\x68\xC3xBC\x64\x61". force_encoding("UTF-8")

    FIGURING IT OUT.
  61. SELECT * from users where id=1; |id|name | ----------- |

    1|Yehüda| DATABASES.
  62. SELECT * from users where id=1; |id|name | ----------- |

    1|Yehüda| SHOW VARIABLES LIKE "character\_set \_database"; |Variable_name |Value| ------------------------------- |character_set_database|utf8 | DATABASES.
  63. SELECT * from users where id=1; |id|name | ----------- |

    1|Yehüda| "\x59\x65\x68\xC3xBC\x64\x61". force_encoding("UTF-8") DATABASES.
  64. BOUNDARIES. rails HTTP MySQL GET params Nokogiri templates POST params

  65. BOUNDARIES. rails UTF-8 HTTP MySQL GET params Nokogiri templates POST

    params
  66. Encoding.default_external = "UTF-8" File.read("/some/file").encoding => Encoding::UTF_8 DEFAULT_EXTERNAL.

  67. Encoding.default_external = "UTF-8" File.read("/some/file").encoding => Encoding::UTF_8 DEFAULT_EXTERNAL. Ruby /some/ le

    UTF-8 UTF-8
  68. Encoding.default_external = "ISO-8859-1" Encoding.default_internal = "UTF-8" File.read("/some/file").encoding => Encoding::UTF_8 DEFAULT_INTERNAL.

  69. Encoding.default_external = "ISO-8859-1" Encoding.default_internal = "UTF-8" File.read("/some/file").encoding => Encoding::UTF_8 DEFAULT_INTERNAL.

    Ruby /some/ le ISO-8859-1 transcode to UTF-8 UTF-8
  70. MULTI-YEAR MISSION.

  71. DRIVERS. MongoDB Redis MySQL2 SQLite3 pg

  72. IN PRACTICE. MySQL2 Driver Ruby MySQL Database database encoding Encoding.default_internal

  73. # For Ruby 1.9, UTF-8 is the default # internal

    and external encoding. Encoding.default_external = "UTF-8" Encoding.default_internal = "UTF-8" # DON'T do this in libraries! RAILTIES/RAILS.RB
  74. TEMPLATES.

  75. # Assume that templates are in the # default_external, which

    defaults to # UTF-8. encoding = Encoding.default_external source.force_encoding(encoding) DEFAULT_EXTERNAL.
  76. # look for a magic encoding comment # this works

    with any template engine! if source.sub!(/\A#{ENCODING_FLAG}/, '') encoding = magic_encoding = $1 else encoding = Encoding.default_external end # Tag the source with the default # external encoding or the encoding # specified in the file source.force_encoding(encoding) EXCEPT...
  77. ENCODING_TAG = Regexp.new( "\\A(<%#{ENCODING_FLAG}-?%>)[ \\t]*") # the handler has its

    own magic comment format def handles_encoding? true end template_source = template.source.dup.force_encoding("BINARY") # find any magic comment and extract the encoding erb = template_source.gsub(ENCODING_TAG, '') encoding = $2 # confirm that the encoding provided is valid and flag it erb.force_encoding valid_encoding( template.source.dup, encoding) # Always make sure we return a String # in the default_internal erb.encode! AND EXCEPT...
  78. # Now, validate that the source we got # back

    from the template handler is valid # in the default_internal. This is for # handlers that handle encoding but screw # up unless source.valid_encoding? raise WrongEncodingError.new( @source, Encoding.default_internal) end VALIDATE IT.
  79. class WrongEncodingError < EncodingError def initialize(string, encoding) @string, @encoding =

    string, encoding end def message @string.force_encoding("BINARY") "Your template was not saved as " \ "valid #{@encoding}. Please either " \ "specify #{@encoding} as the encoding " \ "for your template in your text " \ "editor,or mark the template with its " \ "encoding by inserting the following " \ "as the first line of the template:" \ "\n\n# encoding: <name of correct " \ "encoding>.\n\nThe source of your " \ "template was:\n\n#{@string}" end end CUSTOM EXCEPTION.
  80. <form accept-charset="utf-8"> <textarea> Latin-1 chárâctërs pasted from another application </textarea>

    <button type="submit">Do it!</button> </form> BROWSER DATA.
  81. Possible values: UTF-8 If the user enters characters that are

    not in the character set of the document containing the form, the UTF-8 character set is used. “ MSDN
  82. FAILURE. User changes encoding to Latin-1 User pastes smart quotes

    from Word IE sees that smart quotes are in Latin-1 IE ignores accept-charset :( :( :(
  83. <form accept-charset="utf-8"> <input type="hidden" name="utf8" value="✓"> <textarea> Latin-1 chárâctërs pasted

    from another application </textarea> <button type="submit">Do it!</button> </form> SNOWMAN HACK!
  84. SUCCESS. User changes encoding to Latin-1 User pastes smart quotes

    from Word IE sees that ✓ is not in Latin-1 IE honors accept-charset <3<3<3
  85. IN CONTROL. rails UTF-8 HTTP MySQL GET params Nokogiri templates

    POST params
  86. YOU SHOULD NOT HAVE TO KNOW ANY OF THIS!

  87. WEB APPS. unknown unknowns known knowns known unknowns

  88. WHAT DO YOU WANT TO SPEND YOUR TIME ON?

  89. QUESTIONS? @WYCATS YEHUDAKATZ.COM