Stuck in the Middle with You: Exploring the Connections Between Your App and the Web

Our apps are becoming more complicated and more distributed. We’re extracting APIs and handling callbacks and pings from the services we depend on. We’re using our data and services from different clients, like rich JavaScript applications and mobile apps. And as we fling our logic into more places, it’s harder to see what’s actually going on between them. If you’re working in applications that have become a forest of APIs and services, or you’ve ever said, “I really wish I could just see what kind of data this server thinks I’m handing it, and what I’m getting back,” this talk is for you. With a few tools and some simple techniques, you’ll watch the data go from your apps to your APIs and see your responses, callbacks, and pings come back.

Transcript

1. Stuck in the Middle with You
   Exploring the Connections Between Your
   App and the Web
   Justin Weiss
   @justinweiss
   

   View Slide

2. @justinweiss
   

   View Slide

3. justinweiss.com
   Practicing Rails
   @justinweiss
   

   View Slide

4. Our apps are more
   connected
   @justinweiss
   

   View Slide

5. Phone + API backend
   @justinweiss
   

   View Slide

6. @justinweiss
   

   View Slide

7. Callbacks
   @justinweiss
   

   View Slide

8. @justinweiss
   

   View Slide

9. @justinweiss
   

   View Slide

10. This is fine
    ...right?
    @justinweiss
    

    View Slide

11. Nope.
    @justinweiss
    

    View Slide

12. Feedback? Errors?
    @justinweiss
    

    View Slide

13. JSON::ParserError: 784: unexpected token
    at '
    @justinweiss
    

    View Slide

14. What are the options?
    @justinweiss
    

    View Slide

15. Logging!
    @justinweiss
    

    View Slide

16. Talk over.
    @justinweiss
    

    View Slide

17. It gets
    everywhere
    @justinweiss
    

    View Slide

18. It needs to
    go away
    @justinweiss
    

    View Slide

19. The one thing you didn't
    log is the thing you
    needed to see
    @justinweiss
    

    View Slide

20. Take a step back
    @justinweiss
    

    View Slide

21. It's just
    debugging
    @justinweiss
    

    View Slide

22. What do you need to
    debug effectively?
    @justinweiss
    

    View Slide

23. What is actually true?
    @justinweiss
    

    View Slide

24. You need
    Visibility
    @justinweiss
    

    View Slide

25. 1. That can't happen.
    2. That doesn't happen on my machine.
    3. That shouldn't happen.
    4. Why does that happen?
    5. Oh, I see.
    6. How did that ever work? 1
    1 http://web.archive.org/web/20051027173148/http://www.68k.org/~jrc/old-blog/
    archives/000198.html
    @justinweiss
    

    View Slide

26. What else?
    @justinweiss
    

    View Slide

27. How do you separate?
    @justinweiss
    

    View Slide

28. You need
    Isolation
    @justinweiss
    

    View Slide

29. Visibility
    and
    Isolation
    @justinweiss
    

    View Slide

30. Visibility:
    Requests and Responses
    @justinweiss
    

    View Slide

31. Isolation:
    Use parts independently
    @justinweiss
    

    View Slide

32. What's the dream?
    @justinweiss
    

    View Slide

33. Monitoring of all requests
    without additional code
    @justinweiss
    

    View Slide

34. Can fake out any part
    of the system I own
    @justinweiss
    

    View Slide

35. Three things
    → Send requests
    → Monitor requests and responses
    → Receive requests
    @justinweiss
    

    View Slide

36. That's what this is about
    @justinweiss
    

    View Slide

37. Better debugging ->
    Incremental development
    @justinweiss
    

    View Slide

38. @justinweiss
    

    View Slide

39. Sending
    Requests
    @justinweiss
    

    View Slide

40. curl
    @justinweiss
    

    View Slide

41. $ curl https://api.github.com/users/justinweiss
    {
    "login": "justinweiss",
    "id": 1020,
    "avatar_url": "https://avatars.githubusercontent...",
    ...
    }
    @justinweiss
    

    View Slide

42. $ curl -d "{\"text\": \"hello, _world_\"}" \
    https://api.github.com/markdown
    hello, world
    @justinweiss
    

    View Slide

43. It's everywhere!
    @justinweiss
    

    View Slide

44. @justinweiss
    

    View Slide

45. Postman2
    2 https://www.getpostman.com
    @justinweiss
    

    View Slide

46. @justinweiss
    

    View Slide

47. Paw3
    3 https://paw.cloud
    @justinweiss
    

    View Slide

48. Intercept
    everything!
    @justinweiss
    

    View Slide

49. mitmproxy
    @justinweiss
    

    View Slide

50. What does mitmproxy4 do?
    4 https://mitmproxy.org
    @justinweiss
    

    View Slide

51. brew install mitmproxy
    pip install mitmproxy
    @justinweiss
    

    View Slide

52. @justinweiss
    

    View Slide

53. What about SSL?
    @justinweiss
    

    View Slide

54. @justinweiss
    

    View Slide

55. @justinweiss
    

    View Slide

56. /etc/hosts on mobile
    @justinweiss
    

    View Slide

57. @justinweiss
    

    View Slide

58. Don't forget to
    disconnect!
    @justinweiss
    

    View Slide

59. (Except for the three times it happened while creating this talk)
    @justinweiss
    

    View Slide

60. Alternatives?
    @justinweiss
    

    View Slide

61. Fiddler5
    Charles Proxy6
    6 https://www.charlesproxy.com
    5 http://www.telerik.com/fiddler
    @justinweiss
    

    View Slide

62. Where are we now?
    → Send requests
    @justinweiss
    

    View Slide

63. Where are we now?
    → Send requests
    → Watch requests & responses
    @justinweiss
    

    View Slide

64. Wait for
    feedback
    @justinweiss
    

    View Slide

65. Bidirectional
    communication
    @justinweiss
    

    View Slide

66. Don't call us, we'll call you
    @justinweiss
    

    View Slide

67. Don't call us, we'll call you
    @justinweiss
    

    View Slide

68. requestb.in
    @justinweiss
    

    View Slide

69. @justinweiss
    

    View Slide

70. webmock.io
    @justinweiss
    

    View Slide

71. @justinweiss
    

    View Slide

72. webmock.io
    @justinweiss
    

    View Slide

73. What about local
    development?
    @justinweiss
    

    View Slide

74. ngrok7
    7 https://ngrok.com
    @justinweiss
    

    View Slide

75. @justinweiss
    

    View Slide

76. Where are we now?
    → Send requests
    → Monitor requests and responses
    → Receive requests
    @justinweiss
    

    View Slide

77. What's next?
    @justinweiss
    

    View Slide

78. Example:
    OAuth2
    @justinweiss
    

    View Slide

79. What do we need?
    The client must be capable of interacting with the
    resource owner's user-agent (typically a web
    browser) and capable of receiving incoming
    requests (via redirection) from the authorization
    server.
    — OAuth2 RFC8
    8 https://tools.ietf.org/html/rfc6749#section-4.1
    @justinweiss
    

    View Slide

80. @justinweiss
    

    View Slide

81. → client id
    → client secret
    → redirect uri
    @justinweiss
    

    View Slide

82. @justinweiss
    

    View Slide

83. @justinweiss
    

    View Slide

84. @justinweiss
    

    View Slide

85. Example: Making an
    authenticated request to
    GitHub
    @justinweiss
    

    View Slide

86. @justinweiss
    

    View Slide

87. @justinweiss
    

    View Slide

88. @justinweiss
    

    View Slide

89. @justinweiss
    

    View Slide

90. @justinweiss
    

    View Slide

91. It's complicated
    @justinweiss
    

    View Slide

92. It's complicated
    @justinweiss
    

    View Slide

93. It's complicated
    @justinweiss
    

    View Slide

94. Visibility
    @justinweiss
    

    View Slide

95. Isolation
    @justinweiss
    

    View Slide

96. Visibility
    and
    Isolation
    @justinweiss
    

    View Slide

97. → Send requests: curl, Postman, Paw
    @justinweiss
    

    View Slide

98. → Send requests: curl, Postman, Paw
    → Monitor: mitmproxy, Fiddler, Charles Proxy
    @justinweiss
    

    View Slide

99. → Send requests: curl, Postman, Paw
    → Monitor: mitmproxy, Fiddler, Charles Proxy
    → Receive requests: requestb.in, webmock.io, ngrok
    @justinweiss
    

    View Slide

100. → Send requests: curl, Postman, Paw
     → Monitor: mitmproxy, Fiddler, Charles Proxy
     → Receive requests: requestb.in, webmock.io, ngrok
     @justinweiss
     

     View Slide

101. Justin Weiss
     Avvo
     justin@justinweiss.com
     twitter: @justinweiss
     @justinweiss
     

     View Slide

102. Questions?
     @justinweiss
     

     View Slide

103. Justin Weiss
     Avvo
     justin@justinweiss.com
     twitter: @justinweiss
     @justinweiss
     

     View Slide