Slide 1

Slide 1 text

There are known unknowns... WHY RAILS IS HARD

Slide 2

Slide 2 text

...and unknown unknowns WHY OPEN SOURCE IS HARD

Slide 3

Slide 3 text

[T]here are known knowns; there are things we know we know. We also know there are known unknowns; that is to say we know there are some things we do not know. But there are also unknown unknowns – there are things we do not know we don't know. “ DONALD RUMSFELD

Slide 4

Slide 4 text

OPEN SOURCE. unknown unknowns known knowns known unknowns

Slide 5

Slide 5 text

OPEN SOURCE. normal rails developer

Slide 6

Slide 6 text

OPEN SOURCE. sinatra or node developer

Slide 7

Slide 7 text

OPEN SOURCE. framework developer

Slide 8

Slide 8 text

KNOWN KNOWNS. web browser web server router app

Slide 9

Slide 9 text

KNOWN KNOWNS. web browser passenger / unicorn rails router rails action

Slide 10

Slide 10 text

KNOWN KNOWNS. web browser node server node router node app

Slide 11

Slide 11 text

EQUIVALENT? web browser passenger / unicorn rails router rails action web browser node server node router node app =

Slide 12

Slide 12 text

KNOWN UNKNOWNS

Slide 13

Slide 13 text

CSRF PROTECTION

Slide 14

Slide 14 text

CONNECT.

Slide 15

Slide 15 text

This middleware requires session support, thus should be added somewhere below session() and cookieParser(). “ CONNECT DOCS

Slide 16

Slide 16 text

By default this middleware generates a token named "_csrf" which should be added to requests which mutate state, within a hidden form eld, query-string etc. This token is validated against the visitor's req.session._csrf property. “ CONNECT DOCS

Slide 17

Slide 17 text

NEVER ENLIST END DEVELOPERS IN FRAMEWORK SECURITY.

Slide 18

Slide 18 text

class CashTransferController def create authorize! current_user, params transfer = CashTransfer.new( params[:from_account], params[:to_account], params[:amount] ) queue.push transfer end end WHAT IS CSRF?

Slide 19

Slide 19 text

class CashTransferController def create authorize! current_user, params transfer = CashTransfer.new( params[:from_account], params[:to_account], params[:amount] ) queue.push transfer end end WHAT IS CSRF?

Slide 20

Slide 20 text

rails browser POST /login username: wycats password: <3<3<3<3<3

Slide 21

Slide 21 text

rails browser Set-Cookie: userid=5a73678b7b674

Slide 22

Slide 22 text

Send Money!

Transfer BANK.COM

Slide 23

Slide 23 text

rails browser POST /transfer from=me&to=you&amount=1bi iiiiilliondollars Cookie: userid=5a73678b7b674 automatic POST from bank.com

Slide 24

Slide 24 text

var form = "" + "" + "" + "" + "hehehehe" + ""; $(form).appendTo("body").submit(); EVIL.COM.

Slide 25

Slide 25 text

rails browser POST /transfer from=me&to=you&amount=1bi iiiiilliondollars Cookie: userid=5a73678b7b674 automatic POST from evil.com

Slide 26

Slide 26 text

Send Money!

Transfer MITIGATION.

Slide 27

Slide 27 text

THE GOAL IS DIFFERENTIATING YOUR POST FROM THIRD-PARTY POST.

Slide 28

Slide 28 text

■ On by default ■ Works for all common use-cases ■ HTML forms ■ Ajax requests ■ API clients ■ Not vulnerable to known attacks ■ Earlier "known good" approaches turned out vulnerable to Flash exploit FRAMEWORK GOALS.

Slide 29

Slide 29 text

ON BY DEFAULT.

Slide 30

Slide 30 text

AVOID DISABLING.

Slide 31

Slide 31 text

WORKS FOR ALL COMMON USE CASES.

Slide 32

Slide 32 text

FORMS.

Slide 33

Slide 33 text

100% MANUAL.

Slide 34

Slide 34 text

{% csrf_token %} SEMIAUTOMATIC.

Slide 35

Slide 35 text

<%= form_for @cash_transfer do |f| %> <%= f.text_field :from %> <%= f.text_field :to %> <%= f.text_field :amount %> <%= button "Transfer!" %> <% end %> AUTOMATIC.

Slide 36

Slide 36 text

HTTP METHODS.

Slide 37

Slide 37 text

HEAD SAFE GET SAFE POST UNSAFE PUT UNSAFE DELETE UNSAFE

Slide 38

Slide 38 text

HEAD UNSAFE GET UNSAFE POST UNSAFE PUT UNSAFE DELETE UNSAFE Connect Middleware

Slide 39

Slide 39 text

HEAD SAFE GET SAFE POST UNSAFE PUT SAFE DELETE SAFE Connect Docs

Slide 40

Slide 40 text

POST-ONLY!

Slide 41

Slide 41 text

AJAX REQUESTS. UNSAFE

Slide 42

Slide 42 text

RACK-PROTECTION.

Slide 43

Slide 43 text

AJAX REQUESTS. SAFE

Slide 44

Slide 44 text

SEMIAUTOMATIC.

Slide 45

Slide 45 text

AVOID DISABLING.

Slide 46

Slide 46 text

AVOID DISABLING. I've searched around SO and saw some information about turning off the CSRF check for my view via the csrf_exempt decorator, but I nd that unappealing

Slide 47

Slide 47 text

Generated index.html.erb: <%= csrf_meta_tags %> <%= javascript_include_tag "application" %> Included rails.js: $.ajaxPrefilter(function(options, originalOptions, xhr) { if ( !options.crossDomain ) { rails.CSRFProtection(xhr); } }); rails.CSRFProtection = function(xhr) { var token = $('meta[name="csrf-token"]').attr('content'); if (token) { xhr.setRequestHeader('X-CSRF-Token', token); } }; RAILS.

Slide 48

Slide 48 text

UNKNOWN UNKNOWNS.

Slide 49

Slide 49 text

ENCODINGS.

Slide 50

Slide 50 text

"Yehüda".encode("ISO-8859-1").dump => "Yeh\xFCda" "Yehüda".encode("UTF-8").dump => "Yeh\xC3xBCda" "Yehüda".encode("UTF-16").dump => "\xFE\xFF\x00Y\x00e\x00h\x00\xFC\x00d \x00a" STRINGS.

Slide 51

Slide 51 text

"Yehüda".encode("ISO-8859-1") => "\x59\x65\x68\xFC\x64\x61" "Yehüda".encode("UTF-8") => "\x59\x65\x68\xC3xBC\x64\x61" "Yehüda".encode("UTF-16") => "\xFE\xFF\x00Y\x00e\x00h\x00\xFC\x00d \x00a" STRINGS.

Slide 52

Slide 52 text

# share 7 bits of ASCII "Yehuda".encode("UTF-8").bytes.to_a == "Yehuda".encode("ISO-8859-1").bytes.to_a => true # differ on the remaining bit "Yehüda".encode("UTF-8").bytes.to_a == "Yehüda".encode("ISO-8859-1").bytes.to_a => false UTF-8 VS. LATIN-1.

Slide 53

Slide 53 text

# What does this mean? "\x59\x65\x68\xFC\x64\x61" # Without more metadata, unknown BINARY DATA.

Slide 54

Slide 54 text

# What does this mean? "\x59\x65\x68\xFC\x64\x61" # Without more metadata, unknown # ... we can try to guess ... BINARY DATA.

Slide 55

Slide 55 text

"Yehüda" (latin 1) "Yeh\xFCda" + "Yehüda" (UTF-8) "Yeh\xC3xBCda" = "Yeh\xFCdaYeh\xC3xBCda" MIXING.

Slide 56

Slide 56 text

"Yeh\xFCdaYeh\xC3xBCda" (as Latin-1): YehüdaYehüda INTERPRETING.

Slide 57

Slide 57 text

"Yeh\xFCdaYeh\xC3xBCda" (as Latin-1): YehüdaYehüda "Yeh\xFCdaYeh\xC3xBCda" (as UTF-8): Yeh daYehüda INTERPRETING.

Slide 58

Slide 58 text

A WILD REPLACEMENT CHARACTER APPEARS

Slide 59

Slide 59 text

HTTP 1.1/200 OK Content-Type: application/json; charset=UTF-8 Yehüda Katz "\x59\x65\x68\xC3xBC\x64\x61" FIGURING IT OUT.

Slide 60

Slide 60 text

HTTP 1.1/200 OK Content-Type: application/json; charset=UTF-8 Yehüda Katz "\x59\x65\x68\xC3xBC\x64\x61". force_encoding("UTF-8") FIGURING IT OUT.

Slide 61

Slide 61 text

SELECT * from users where id=1; |id|name | ----------- | 1|Yehüda| DATABASES.

Slide 62

Slide 62 text

SELECT * from users where id=1; |id|name | ----------- | 1|Yehüda| SHOW VARIABLES LIKE "character\_set \_database"; |Variable_name |Value| ------------------------------- |character_set_database|utf8 | DATABASES.

Slide 63

Slide 63 text

SELECT * from users where id=1; |id|name | ----------- | 1|Yehüda| "\x59\x65\x68\xC3xBC\x64\x61". force_encoding("UTF-8") DATABASES.

Slide 64

Slide 64 text

BOUNDARIES. rails HTTP MySQL GET params Nokogiri templates POST params

Slide 65

Slide 65 text

BOUNDARIES. rails UTF-8 HTTP MySQL GET params Nokogiri templates POST params

Slide 66

Slide 66 text

Encoding.default_external = "UTF-8" File.read("/some/file").encoding => Encoding::UTF_8 DEFAULT_EXTERNAL.

Slide 67

Slide 67 text

Encoding.default_external = "UTF-8" File.read("/some/file").encoding => Encoding::UTF_8 DEFAULT_EXTERNAL. Ruby /some/ le UTF-8 UTF-8

Slide 68

Slide 68 text

Encoding.default_external = "ISO-8859-1" Encoding.default_internal = "UTF-8" File.read("/some/file").encoding => Encoding::UTF_8 DEFAULT_INTERNAL.

Slide 69

Slide 69 text

Encoding.default_external = "ISO-8859-1" Encoding.default_internal = "UTF-8" File.read("/some/file").encoding => Encoding::UTF_8 DEFAULT_INTERNAL. Ruby /some/ le ISO-8859-1 transcode to UTF-8 UTF-8

Slide 70

Slide 70 text

MULTI-YEAR MISSION.

Slide 71

Slide 71 text

DRIVERS. MongoDB Redis MySQL2 SQLite3 pg

Slide 72

Slide 72 text

IN PRACTICE. MySQL2 Driver Ruby MySQL Database database encoding Encoding.default_internal

Slide 73

Slide 73 text

# For Ruby 1.9, UTF-8 is the default # internal and external encoding. Encoding.default_external = "UTF-8" Encoding.default_internal = "UTF-8" # DON'T do this in libraries! RAILTIES/RAILS.RB

Slide 74

Slide 74 text

TEMPLATES.

Slide 75

Slide 75 text

# Assume that templates are in the # default_external, which defaults to # UTF-8. encoding = Encoding.default_external source.force_encoding(encoding) DEFAULT_EXTERNAL.

Slide 76

Slide 76 text

# look for a magic encoding comment # this works with any template engine! if source.sub!(/\A#{ENCODING_FLAG}/, '') encoding = magic_encoding = $1 else encoding = Encoding.default_external end # Tag the source with the default # external encoding or the encoding # specified in the file source.force_encoding(encoding) EXCEPT...

Slide 77

Slide 77 text

ENCODING_TAG = Regexp.new( "\\A(<%#{ENCODING_FLAG}-?%>)[ \\t]*") # the handler has its own magic comment format def handles_encoding? true end template_source = template.source.dup.force_encoding("BINARY") # find any magic comment and extract the encoding erb = template_source.gsub(ENCODING_TAG, '') encoding = $2 # confirm that the encoding provided is valid and flag it erb.force_encoding valid_encoding( template.source.dup, encoding) # Always make sure we return a String # in the default_internal erb.encode! AND EXCEPT...

Slide 78

Slide 78 text

# Now, validate that the source we got # back from the template handler is valid # in the default_internal. This is for # handlers that handle encoding but screw # up unless source.valid_encoding? raise WrongEncodingError.new( @source, Encoding.default_internal) end VALIDATE IT.

Slide 79

Slide 79 text

class WrongEncodingError < EncodingError def initialize(string, encoding) @string, @encoding = string, encoding end def message @string.force_encoding("BINARY") "Your template was not saved as " \ "valid #{@encoding}. Please either " \ "specify #{@encoding} as the encoding " \ "for your template in your text " \ "editor,or mark the template with its " \ "encoding by inserting the following " \ "as the first line of the template:" \ "\n\n# encoding: .\n\nThe source of your " \ "template was:\n\n#{@string}" end end CUSTOM EXCEPTION.

Slide 80

Slide 80 text

Latin-1 chárâctërs pasted from another application Do it! BROWSER DATA.

Slide 81

Slide 81 text

Possible values: UTF-8 If the user enters characters that are not in the character set of the document containing the form, the UTF-8 character set is used. “ MSDN

Slide 82

Slide 82 text

FAILURE. User changes encoding to Latin-1 User pastes smart quotes from Word IE sees that smart quotes are in Latin-1 IE ignores accept-charset :( :( :(

Slide 83

Slide 83 text

Latin-1 chárâctërs pasted from another application Do it! SNOWMAN HACK!

Slide 84

Slide 84 text

SUCCESS. User changes encoding to Latin-1 User pastes smart quotes from Word IE sees that ✓ is not in Latin-1 IE honors accept-charset <3<3<3

Slide 85

Slide 85 text

IN CONTROL. rails UTF-8 HTTP MySQL GET params Nokogiri templates POST params

Slide 86

Slide 86 text

YOU SHOULD NOT HAVE TO KNOW ANY OF THIS!

Slide 87

Slide 87 text

WEB APPS. unknown unknowns known knowns known unknowns

Slide 88

Slide 88 text

WHAT DO YOU WANT TO SPEND YOUR TIME ON?

Slide 89

Slide 89 text

QUESTIONS? @WYCATS YEHUDAKATZ.COM