Twitch's GraphQL Transformation

Twitch has over six hundred engineers, millions of concurrent viewers, and is one of the highest trafficked sites on the internet today. Our API handles hundreds of thousands of requests per second and powers many different clients. The API aggregates hundreds of underlying services together in a singular, (hopefully) coherent package.

Adopting a GraphQL API enabled us to rewrite our aging web and mobile clients within a few months, giving users a much snappier experience. We’ve built tooling to optimize for developer velocity on the service side as well. Code generation keeps the cost of adding types and fields minimal, and automated integration tests ensure changes are safe. We’ve wrangled dataloaders into shape to provide efficient, high-performance operations across a multitude of backing services.

And perhaps most importantly, GraphQL has acted as a guiding force to a more standardized and flexible service ecosystem. We’ve been pushed to reconsider past decisions around service aspects like authorization and pagination, and have come out with much improved systems.

Presented at GraphQL Summit 2017.

Transcript

1. Our GraphQL Transformation
2. tonyghita tonyghita @tonyghita
5. Our GraphQL Transformation
6. social video
9. paradigm shift
10. GET /users/:id/friends query getFriends { currentUser { friends { name } } }
11. missing unit test
12. improved services
14. 600+ nerds
15. api gateway /users /streams /games /friends service service service endpoints
16. api gateway /users /streams /games /friends service service service endpoints 100+
17. api gateway /users /streams /games /friends service service MONOLITH endpoints 100+
18. api gateway /users /streams /games /friends service service MONOLITH endpoints 100+
19. api gateway /users /streams /games /friends service service MONOLITH endpoints 100+
20. REST, JSON
21. sum m it 2016 sum m it 2017 proof-of-concept type com position & pagination authN / authZ ??? production beta PROTOTYPE PRODUCTION
23. sum m it 2017 proof-of-concept type com position & pagination authN / authZ ??? live beta PROTOTYPE PRODUCTION sum m it 2016
25. sum m it 2016 sum m it 2017 type com position & pagination authN / authZ ??? production beta PROTOTYPE PRODUCTION proof-of-concept
26. – Cruise Industry News Quarterly, 1999 “... if Henry Ford canvassed people on whether or not he should build a motor car, they'd probably tell him what they really wanted was a faster horse”
27. type Query { users(ids: [ID!], names: [String!]): [User] } type User { id: ID name: String }
28. resolve types & fields load data efficiently integrate services
29. sum m it 2016 sum m it 2017 proof-of-concept authN / authZ ??? production beta PROTOTYPE PRODUCTION type com position & pagination
30. streams games
31. streams games users
32. type Query { users(ids: [ID!], names: [String!]): [User] streams(first: Int = 10): [Stream] games(first: Int = 10): [Game] } type User { id: ID name: String } type Stream { id: ID broadcaster: User game: Game name: String viewers: Int } type Game { id: ID name: String viewers: Int }
33. type composition pagination
34. pagination
35. offset + limit cursors timestamp
36. REST /v1 /v2 /v3 /v4 /v5
37. "a01s4=="
38. type Query { games(first: Int, after: Cursor): GameConnection } query getFeaturedGames { games(first: 10, after: "a01s4 ==") { edges { cursor
 node { id, name, viewersCount } } } }
39. standardize the interface
40. relay cursor connection spec bit.ly/gqlPage
41. sum m it 2016 sum m it 2017 proof-of-concept type com position & pagination authN / authZ ??? production beta PROTOTYPE PRODUCTION
42. "auth"
43. api gateway endpoints /users /streams /games /friends service service service
44. authentication api gateway authorization authorization authorization authorization service service service endpoints
45. authentication api gateway authorization authorization authorization authorization service service service endpoints
46. authentication api gateway authorization authorization authorization authorization service service service endpoints
47. authentication api gateway service service service endpoint /graphql
48. authentication api gateway service service service endpoint /graphql authorization?
49. authentication api gateway service service service endpoint /graphql authorization?
50. authentication api gateway service service service endpoint /graphql authorization? authorization? authorization? authorization?
51. authentication api gateway service service service endpoint /graphql authorization? authorization? authorization? authorization?
53. authentication api gateway service service service endpoint /graphql authorization authorization authorization
54. separation of concerns push logic to services
55. easy
56. :\
57. ;)
58. keep it simple
59. bit.ly/gqlAuth
60. sum m it 2016 sum m it 2017 proof-of-concept type com position & pagination authN / authZ production beta PROTOTYPE PRODUCTION ???
61. Time Contributors Me 50+ contributors
62. Time Contributors Me 50+ contributors
63. what i wish i knew: scaling contributions
64. document good practices
65. iterate on a styleguide
66. write linters
67. boilerplate
68. type User { id: ID! name: String! } func (r *UserResolver) ID() (graphql.ID, error) { user, err := r.loadFn() if err != nil { return graphql.ID(""), err } if user == nil { return graphql.ID(""), errors.New("not found") } return graphql.ID(user.ID), nil } func (r *UserResolver) Name() (string, error) { user, err := r.loadFn() if err != nil { return "", err } if user == nil { return "", errors.New("not found") } return user.Name, nil } type UserResolver struct { loadFn *dataloader.Thunk }
69. type User { id: ID! name: String! } func (r *UserResolver) ID() (graphql.ID, error) { user, err := r.loadFn() if err != nil { return graphql.ID(""), err } if user == nil { return graphql.ID(""), errors.New("not found") } return graphql.ID(user.ID), nil } func (r *UserResolver) Name() (string, error) { user, err := r.loadFn() if err != nil { return "", err } if user == nil { return "", errors.New("not found") } return user.Name, nil } type UserResolver struct { loadFn *dataloader.Thunk }
70. type User { id: ID! name: String! } func (r *UserResolver) ID() (graphql.ID, error) { user, err := r.loadFn() if err != nil { return graphql.ID(""), err } if user == nil { return graphql.ID(""), errors.New("not found") } return graphql.ID(user.ID), nil } func (r *UserResolver) Name() (string, error) { user, err := r.loadFn() if err != nil { return "", err } if user == nil { return "", errors.New("not found") } return user.Name, nil } type UserResolver struct { loadFn *dataloader.Thunk }
71. generate code
72. type User { id: ID! name: String! } func (r *UserResolver) ID() (graphql.ID, error) { user, err := r.loadFn() if err != nil { return graphql.ID(""), err } if user == nil { return graphql.ID(""), errors.New("not found") } return graphql.ID(user.ID), nil } func (r *UserResolver) Name() (string, error) { user, err := r.loadFn() if err != nil { return "", err } if user == nil { return "", errors.New("not found") } return user.Name, nil } type UserResolver struct { loadFn *dataloader.Thunk }
73. automate knowledge share
74. – African Proverb “If you want to go fast, go alone. If you want to go far, go together.”
75. go.twitch.tv
76. what i wish i knew: nullability & errors BONUS
77. sum m it 2016 sum m it 2017 proof-of-concept type com position & pagination authN / authZ PROTOTYPE PRODUCTION ??? production beta
78. nullability
79. query getNullabilityFail { user(name: "does not exist") { followers { nodes { id, name } } } } type Query { user(name: String): User }
80. query getNullabilityFail { user(name: "does not exist") { followers { nodes { id, name } } } } { "data": { "user": { "followers": { "nodes": [] }
 } } }
81. query getNullabilityFail { user(name: "does not exist") { followers { nodes { id, name } } } } { "data": { "user": { "followers": { "nodes": [] }
 } } } X
82. query getNullabilityFail { user(name: "does not exist") { followers { nodes { id, name } } } } { "data": { "user": null } }
83. errors
84. be lazy
85. assume
86. mutations bit.ly/gqlMutate
87. next?
88. thanks!