HTTP in a Hostile World (Droidcon Toronto 2019)

69252b3de5cb7f464c09301d9a6b0401?s=47 Jesse Wilson
November 13, 2019

HTTP in a Hostile World (Droidcon Toronto 2019)

Video: https://youtu.be/tPA9n2mgClI

Networking on Android is difficult work. We need to display fresh data and rich media while limiting our network use. Unfortunately cookie-cutter solutions are not acceptable.

In this talk we’ll:

🥌 Verify that you’re using HTTPS securely
🥌 Acknowledge possible connectivity problems
🥌 Get performance tips
🥌 Code for maintainable APIs
🥌 Mock programmers that don’t test their networking

The body of this talk will be using HTTP effectively.

69252b3de5cb7f464c09301d9a6b0401?s=128

Jesse Wilson

November 13, 2019
Tweet

Transcript

  1. 2.

    HTTP? • Web Pages • Web Resources (images!) • Downloads

    • Web APIs • Mobile APIs • Server APIs • Core Networking APIs (DNS) • Pretty much all networking ever
  2. 8.

    POST /pizza HTTP/1.1 host: pizza.app content-length: 50 content-type: application/json; charset=UTF-8

    {x "size": "large", "toppings": ["pineapple"] }x HTTP/1.1 200 OK content-length: 21 
 content-type: application/json; charset=UTF-8 {x "time": “30m", "price": 15.00 }x Response
  3. 9.

    HTTP/1.1 200 OK content-length: 21 
 content-type: application/json; charset=UTF-8 {x

    "time": “30m", "price": 15.00 }x POST /pizza HTTP/1.1 host: pizza.app content-length: 50 content-type: application/json; charset=UTF-8 {x "size": "large", "toppings": ["pineapple"] }x Status
  4. 10.

    HTTP/1.1 200 OK content-length: 21 
 content-type: application/json; charset=UTF-8 {x

    "time": “30m", "price": 15.00 }x POST /pizza HTTP/1.1 host: pizza.app content-length: 50 content-type: application/json; charset=UTF-8 {x "size": "large", "toppings": ["pineapple"] }x Headers
  5. 11.

    HTTP/1.1 200 OK content-length: 21 
 content-type: application/json; charset=UTF-8 {x

    "time": “30m", "price": 15.00 }x POST /pizza HTTP/1.1 host: pizza.app content-length: 50 content-type: application/json; charset=UTF-8 {x "size": "large", "toppings": ["pineapple"] }x Body
  6. 12.

    HTTP/1.1 200 OK content-length: 21 
 content-type: application/json; charset=UTF-8 {x

    "time": “30m", "price": 15.00 }x POST /pizza HTTP/1.1 host: pizza.app content-length: 50 content-type: application/json; charset=UTF-8 {x "size": "large", "toppings": ["pineapple"] }x
  7. 13.

    $ nc square.com 80 GET /robots.txt HTTP/1.1 host: square.com HTTP/1.1

    301 Moved Permanently location: https://square.com/robots.txt date: Thu, 07 Nov 2019 04:05:21 GMT server: envoy content-length: 0
 $
  8. 14.

    HTTP/0.9 HTTP/1.0 HTTP/1.1 HTTP/2 HTTP/3 1991 1996 1997 2015 2019

    HTML only headers and status codes persistent connections multiplexing and binary encoding out-of-order packets and IP address changes
  9. 15.

    HTTP in a Hostile World? • Correct • Ready •

    Secure • New stuff I’m excited about
  10. 16.
  11. 22.

    What Triggers Repeats? • Users double-tapping the send button •

    Applications retrying • Network code retrying • Gateways retrying
  12. 23.

    Implementing Idempotence: Identify Intention • Generate a universally unique token

    in the UI:
 d6abe0fcd5d8c51e6c8a9159a873efcc • Write the outbound intention to a persistent queue, like your app’s SQLite database or your cloud queue service (SQS, etc.) • When processing an action, persist the identifier • The easy way: create a UNIQUE INDEX on the token
  13. 25.

    API Contract v. 1 • Client sends a pizza order

    in JSON • Server sends HTTP 200 and the delivery info in JSON
  14. 26.

    HTTP/1.1 200 OK { "time": "30m", "price": 15.00 } POST

    /pizza HTTP/1.1 { "size": "large", "toppings": ["pineapple"] }
  15. 27.

    API Contract v. 2 • Client sends a pizza order

    in JSON • Success: Server sends HTTP 200 and the delivery info in JSON • Failure: Server sends HTTP 500 and the failure info in JSON
  16. 31.

    Designing HTTP APIs • Conceptually, your client talks to your

    server • In practice, ‘invisible’ infrastructure exists • Sometimes it fails to be invisible
  17. 32.

    Status Codes just a sec thumbs up! response is elsewhere

    client problem server problem 1xx 2xx 3xx 4xx 5xx
  18. 33.

    Resilient to Middle Boxes • Send Content-Type on outbound requests

    • Expect Content-Type on inbound responses • HTTP 200? All Good • Other status codes? Be defensive!
  19. 34.

    REST: Retrofit interface PizzaShop { @GET("/{store}/{date}/deliveries") fun deliveries( @Path("store") store:

    String, @Path("date") date: String ): Call<List<Delivery>> } square.github.io/retrofit
  20. 36.

    interface PizzaShopClient : Service { @WireRpc( path = "/yum.pizza.PizzaShop/OrderFeed", requestAdapter

    = "yum.pizza.Update#ADAPTER”, responseAdapter = "yum.pizza.Order#ADAPTER" ) fun OrderFeed(): GrpcStreamingCall<Update, Order> } gRPC: Wire 3 square.github.io/wire
  21. 38.

    Testing: MockWebServer github.com/square/okhttp/mockwebserver val server = MockWebServer() server.start() server.enqueue(MockResponse() .setHeader("Cookie",

    "fave_topping=pineapple") .setBody("""{"time": "30m", "price": 19.99 }""")) val serverUrl = server.url("/")
  22. 41.
  23. 42.
  24. 44.

    Connectivity is a Lie • Bell and Rogers fail •

    Backends fail • Graceful fallbacks are possible
  25. 49.

    Good-Enough Caching • It may change
 Cache-Control: max-age=0 • It

    will never change
 Cache-Control: max-age=31536000
  26. 50.

    val cache = Cache( directory = File(context.cacheDir, "http"), maxSize =

    100L * 1024 * 1024 ) val client = OkHttpClient.Builder() .cache(cache) .build()
  27. 51.

    Compression • Make sure gzip responses are enabled on client

    and server • Cuts size by 70% for JSON! • Consider gzip requests too • Use brotli for best compression
  28. 52.

    CDNs • Content Delivery Networks • Easiest way to improve

    upon speed of light • Great for availability • These can do image resizing!
  29. 53.

    Granularity • HTTP/1’s per-resource overhead is high • More speed?


    Fewer resources! • With HTTP/2 logical resources are best whatsapp.com
  30. 54.
  31. 55.

    Data is Dangerous • You’re keeping other people’s secrets •

    You don’t really know what’s secret and what’s not • Attackers are persistent
  32. 57.

    Limit Exposure to Data • Are we collecting this field

    because we need it? • When will we delete it? • GDPR in Europe, CCPA in California • Differential Privacy
  33. 58.
  34. 59.

    TLS: Necessary • Transport Layer Security is the s in

    https:// • Confirms that you’re really talking to https://cash.app/ • Prevents eavesdropping • Prevents interference
  35. 60.

    TLS: Not Sufficient • Doesn’t impact how https://cash.app/ secures the

    data • Doesn’t prevent URLs like https://cash.app.attacker.com/ • Old implementations of TLS had severe bugs • Nation-state attackers do nasty things
  36. 61.
  37. 62.

    Nation-State Attacks • A certificate authority is a company that

    confirms ownership of https://cash.app/ and signs our TLS certificate • By default browsers trust ~300 certificate authorities (CAs) • Nation states add themselves to the browser • Or compromise a vulnerable certificate authority
  38. 63.
  39. 65.

    CertificatePinner certificatePinner = new CertificatePinner.Builder() .add("pizza.yum", "sha256/9TxoNDzGwoGUV8oDSZI8XpGpwyPz0GY/xAyWRLLjpPc=") .add("pizza.yum", "sha256/36Bv98D5O9bYGYRXnMTZJLQRW8i8fvOiWC/TJ4jOhLw=") .add("pizza.yum",

    "sha256/++iJU8rCHLXviXyKrJ2yqBWjs/5+LAqWuP04dHVHFb0=") .build(); OkHttpClient okHttpClient = new OkHttpClient.Builder() .certificatePinner(certificatePinner) .build();
  40. 66.

    TLS Evolution first widely-used release renamed from SSL to TLS

    bug fixes & new crypto bug fixes & new crypto major update with simplifications! 1996 1999 2006 2008 2018 SSLv3 TLSv1 TLSv1.1 TLSv1.2 TLSv1.3
  41. 67.

    TLS Evolution first widely-used release renamed from SSL to TLS

    bug fixes & new crypto bug fixes & new crypto major update with simplifications! JAN 2020 1996 1999 2006 2008 2018 SSLv3 TLSv1 TLSv1.1 TLSv1.2 TLSv1.3
  42. 68.

    TLS Evolution first widely-used release renamed from SSL to TLS

    bug fixes & new crypto bug fixes & new crypto major update with simplifications! JAN 2020 1996 1999 2006 2008 2018 Android 5+ Or Google Play Security Provider SSLv3 TLSv1 TLSv1.1 TLSv1.2 TLSv1.3
  43. 69.

    Whitebox Attackers • Our apps are instruction manuals for reverse

    engineers • There are amazing tools that make this point-and-click easy
  44. 71.

    Defending Against
 Whitebox Attackers • A sufficiently-determined attacker will succeed

    • All we can do is make it really annoying • Perhaps a magic HTTP header?
  45. 74.
  46. 75.
  47. 76.

    HTTP/3 • Better worst-case latency • Change your IP address

    without restarting downloads! quicwg.org
  48. 78.

    TCP streams Requests and Responses TCP packets HTTP/1.1 TCP streams

    TLS streams Requests and Responses TCP packets HTTP/1.1 with TLS
  49. 79.

    TCP streams Requests and Responses TCP packets HTTP/1.1 TCP streams

    TLS streams Requests and Responses TCP packets HTTP/1.1 with TLS TCP streams TLS streams HTTP/2 frames TCP packets HTTP/2 with TLS Requests and Responses
  50. 80.

    TCP streams Requests and Responses TCP packets HTTP/1.1 TCP streams

    TLS streams Requests and Responses TCP packets HTTP/1.1 with TLS TCP streams TLS streams HTTP/2 frames TCP packets HTTP/2 with TLS Requests and Responses UDP packets HTTP/3 with TLS Requests and Responses QUIC streams
  51. 81.

    • What if Retrofit inspired a server framework? • gRPC

    + REST in one framework • Early days! Watch this space cashapp.github.io/misk
  52. 82.