Why Rails is Hard

There are known unknowns... WHY RAILS IS HARD

...and unknown unknowns WHY OPEN SOURCE IS HARD

[T]here are known knowns; there are things we know we
know. We also know there are known unknowns; that is to say we know there are some things we do not know. But there are also unknown unknowns – there are things we do not know we don't know. “ DONALD RUMSFELD

OPEN SOURCE. unknown unknowns known knowns known unknowns

OPEN SOURCE. normal rails developer

OPEN SOURCE. sinatra or node developer

OPEN SOURCE. framework developer

KNOWN KNOWNS. web browser web server router app

KNOWN KNOWNS. web browser passenger / unicorn rails router rails
action

KNOWN KNOWNS. web browser node server node router node app

EQUIVALENT? web browser passenger / unicorn rails router rails action
web browser node server node router node app =

KNOWN UNKNOWNS

CSRF PROTECTION

CONNECT.

This middleware requires session support, thus should be added somewhere
below session() and cookieParser(). “ CONNECT DOCS

By default this middleware generates a token named "_csrf" which
should be added to requests which mutate state, within a hidden form eld, query-string etc. This token is validated against the visitor's req.session._csrf property. “ CONNECT DOCS

NEVER ENLIST END DEVELOPERS IN FRAMEWORK SECURITY.

class CashTransferController def create authorize! current_user, params transfer = CashTransfer.new(
params[:from_account], params[:to_account], params[:amount] ) queue.push transfer end end WHAT IS CSRF?

rails browser POST /login username: wycats password: <3<3<3<3<3

rails browser Set-Cookie: userid=5a73678b7b674

<html> <h1>Send Money!</h1> <form action='bank.com/transfer'> <input type='text' name='from' value='me'> <input
type='text' name='to' value='you'> <input type='text' name='amount' value='1biiiiiilliondollars'> <button type="submit">Transfer</button> </form> </html> BANK.COM

rails browser POST /transfer from=me&to=you&amount=1bi iiiiilliondollars Cookie: userid=5a73678b7b674 automatic POST
from bank.com

var form = "<form action='bank.com/transfer'>" + "<input type='text' name='from' value='me'>"
+ "<input type='text' name='to' value='you'>" + "<input type='text' name='amount' value='1biiiiiilliondollars'>" + "<button type="submit">hehehehe</button>" + "</form>"; $(form).appendTo("body").submit(); EVIL.COM.

rails browser POST /transfer from=me&to=you&amount=1bi iiiiilliondollars Cookie: userid=5a73678b7b674 automatic POST
from evil.com

<html> <h1>Send Money!</h1> <form action='bank.com/transfer'> <input type='hidden' name='csrf_token' value='b674005056434a48054707d'> <input
type='text' name='from' value='me'> <input type='text' name='to' value='you'> <input type='text' name='amount' value='1biiiiiilliondollars'> <button type="submit">Transfer</button> </form> </html> MITIGATION.

THE GOAL IS DIFFERENTIATING YOUR POST FROM THIRD-PARTY POST.

▪ On by default ▪ Works for all common use-cases
▪ HTML forms ▪ Ajax requests ▪ API clients ▪ Not vulnerable to known attacks ▪ Earlier "known good" approaches turned out vulnerable to Flash exploit FRAMEWORK GOALS.

ON BY DEFAULT.

AVOID DISABLING.

WORKS FOR ALL COMMON USE CASES.

FORMS.

100% MANUAL.

<form action="/cash_transfer"> {% csrf_token %} SEMIAUTOMATIC.

<%= form_for @cash_transfer do |f| %> <%= f.text_field :from %>
<%= f.text_field :to %> <%= f.text_field :amount %> <%= button "Transfer!" %> <% end %> AUTOMATIC.

HTTP METHODS.

HEAD SAFE GET SAFE POST UNSAFE PUT UNSAFE DELETE UNSAFE

HEAD UNSAFE GET UNSAFE POST UNSAFE PUT UNSAFE DELETE UNSAFE
Connect Middleware

HEAD SAFE GET SAFE POST UNSAFE PUT SAFE DELETE SAFE
Connect Docs

POST-ONLY!

AJAX REQUESTS. UNSAFE

RACK-PROTECTION.

AJAX REQUESTS. SAFE

SEMIAUTOMATIC.

AVOID DISABLING.

AVOID DISABLING. I've searched around SO and saw some information
about turning off the CSRF check for my view via the csrf_exempt decorator, but I nd that unappealing

Generated index.html.erb: <head> <%= csrf_meta_tags %> <%= javascript_include_tag "application" %>
</head> Included rails.js: $.ajaxPrefilter(function(options, originalOptions, xhr) { if ( !options.crossDomain ) { rails.CSRFProtection(xhr); } }); rails.CSRFProtection = function(xhr) { var token = $('meta[name="csrf-token"]').attr('content'); if (token) { xhr.setRequestHeader('X-CSRF-Token', token); } }; RAILS.

UNKNOWN UNKNOWNS.

ENCODINGS.

"Yehüda".encode("ISO-8859-1").dump => "Yeh\xFCda" "Yehüda".encode("UTF-8").dump => "Yeh\xC3xBCda" "Yehüda".encode("UTF-16").dump => "\xFE\xFF\x00Y\x00e\x00h\x00\xFC\x00d \x00a"
STRINGS.

"Yehüda".encode("ISO-8859-1") => "\x59\x65\x68\xFC\x64\x61" "Yehüda".encode("UTF-8") => "\x59\x65\x68\xC3xBC\x64\x61" "Yehüda".encode("UTF-16") => "\xFE\xFF\x00Y\x00e\x00h\x00\xFC\x00d \x00a"
STRINGS.

# share 7 bits of ASCII "Yehuda".encode("UTF-8").bytes.to_a == "Yehuda".encode("ISO-8859-1").bytes.to_a =>
true # differ on the remaining bit "Yehüda".encode("UTF-8").bytes.to_a == "Yehüda".encode("ISO-8859-1").bytes.to_a => false UTF-8 VS. LATIN-1.

# What does this mean? "\x59\x65\x68\xFC\x64\x61" # Without more metadata,
unknown BINARY DATA.

# What does this mean? "\x59\x65\x68\xFC\x64\x61" # Without more metadata,
unknown # ... we can try to guess ... BINARY DATA.

"Yehüda" (latin 1) "Yeh\xFCda" + "Yehüda" (UTF-8) "Yeh\xC3xBCda" = "Yeh\xFCdaYeh\xC3xBCda"
MIXING.

"Yeh\xFCdaYeh\xC3xBCda" (as Latin-1): YehüdaYehÃ¼da INTERPRETING.

"Yeh\xFCdaYeh\xC3xBCda" (as Latin-1): YehüdaYehÃ¼da "Yeh\xFCdaYeh\xC3xBCda" (as UTF-8): Yeh daYehüda INTERPRETING.

A WILD REPLACEMENT CHARACTER APPEARS

HTTP 1.1/200 OK Content-Type: application/json; charset=UTF-8 Yehüda Katz "\x59\x65\x68\xC3xBC\x64\x61" FIGURING
IT OUT.

HTTP 1.1/200 OK Content-Type: application/json; charset=UTF-8 Yehüda Katz "\x59\x65\x68\xC3xBC\x64\x61". force_encoding("UTF-8")
FIGURING IT OUT.

SELECT * from users where id=1; |id|name | ----------- |
1|Yehüda| DATABASES.

1|Yehüda| SHOW VARIABLES LIKE "character\_set \_database"; |Variable_name |Value| ------------------------------- |character_set_database|utf8 | DATABASES.

1|Yehüda| "\x59\x65\x68\xC3xBC\x64\x61". force_encoding("UTF-8") DATABASES.

BOUNDARIES. rails HTTP MySQL GET params Nokogiri templates POST params

BOUNDARIES. rails UTF-8 HTTP MySQL GET params Nokogiri templates POST
params

Encoding.default_external = "UTF-8" File.read("/some/file").encoding => Encoding::UTF_8 DEFAULT_EXTERNAL.

Encoding.default_external = "UTF-8" File.read("/some/file").encoding => Encoding::UTF_8 DEFAULT_EXTERNAL. Ruby /some/ le
UTF-8 UTF-8

Encoding.default_external = "ISO-8859-1" Encoding.default_internal = "UTF-8" File.read("/some/file").encoding => Encoding::UTF_8 DEFAULT_INTERNAL.

Encoding.default_external = "ISO-8859-1" Encoding.default_internal = "UTF-8" File.read("/some/file").encoding => Encoding::UTF_8 DEFAULT_INTERNAL.
Ruby /some/ le ISO-8859-1 transcode to UTF-8 UTF-8

MULTI-YEAR MISSION.

DRIVERS. MongoDB Redis MySQL2 SQLite3 pg

IN PRACTICE. MySQL2 Driver Ruby MySQL Database database encoding Encoding.default_internal

# For Ruby 1.9, UTF-8 is the default # internal
and external encoding. Encoding.default_external = "UTF-8" Encoding.default_internal = "UTF-8" # DON'T do this in libraries! RAILTIES/RAILS.RB

TEMPLATES.

# Assume that templates are in the # default_external, which
defaults to # UTF-8. encoding = Encoding.default_external source.force_encoding(encoding) DEFAULT_EXTERNAL.

# look for a magic encoding comment # this works
with any template engine! if source.sub!(/\A#{ENCODING_FLAG}/, '') encoding = magic_encoding = $1 else encoding = Encoding.default_external end # Tag the source with the default # external encoding or the encoding # specified in the file source.force_encoding(encoding) EXCEPT...

ENCODING_TAG = Regexp.new( "\\A(<%#{ENCODING_FLAG}-?%>)[ \\t]*") # the handler has its
own magic comment format def handles_encoding? true end template_source = template.source.dup.force_encoding("BINARY") # find any magic comment and extract the encoding erb = template_source.gsub(ENCODING_TAG, '') encoding = $2 # confirm that the encoding provided is valid and flag it erb.force_encoding valid_encoding( template.source.dup, encoding) # Always make sure we return a String # in the default_internal erb.encode! AND EXCEPT...

# Now, validate that the source we got # back
from the template handler is valid # in the default_internal. This is for # handlers that handle encoding but screw # up unless source.valid_encoding? raise WrongEncodingError.new( @source, Encoding.default_internal) end VALIDATE IT.

class WrongEncodingError < EncodingError def initialize(string, encoding) @string, @encoding =
string, encoding end def message @string.force_encoding("BINARY") "Your template was not saved as " \ "valid #{@encoding}. Please either " \ "specify #{@encoding} as the encoding " \ "for your template in your text " \ "editor,or mark the template with its " \ "encoding by inserting the following " \ "as the first line of the template:" \ "\n\n# encoding: <name of correct " \ "encoding>.\n\nThe source of your " \ "template was:\n\n#{@string}" end end CUSTOM EXCEPTION.

<form accept-charset="utf-8"> <textarea> Latin-1 chárâctërs pasted from another application </textarea>
<button type="submit">Do it!</button> </form> BROWSER DATA.

Possible values: UTF-8 If the user enters characters that are
not in the character set of the document containing the form, the UTF-8 character set is used. “ MSDN

FAILURE. User changes encoding to Latin-1 User pastes smart quotes
from Word IE sees that smart quotes are in Latin-1 IE ignores accept-charset :( :( :(

<form accept-charset="utf-8"> <input type="hidden" name="utf8" value="✓"> <textarea> Latin-1 chárâctërs pasted
from another application </textarea> <button type="submit">Do it!</button> </form> SNOWMAN HACK!

SUCCESS. User changes encoding to Latin-1 User pastes smart quotes
from Word IE sees that ✓ is not in Latin-1 IE honors accept-charset <3<3<3

IN CONTROL. rails UTF-8 HTTP MySQL GET params Nokogiri templates
POST params

YOU SHOULD NOT HAVE TO KNOW ANY OF THIS!

WEB APPS. unknown unknowns known knowns known unknowns

WHAT DO YOU WANT TO SPEND YOUR TIME ON?

QUESTIONS? @WYCATS YEHUDAKATZ.COM

Why Rails is Hard

Why Rails is Hard

More Decks by Yehuda Katz

Other Decks in Technology

Featured

Transcript