Real world Graphene

Graphene is currently the most popular framework for building a GraphQL in Python and it’s also an obvious choice for adding a GraphQL layer to Django applications. Over the course of a year, we successfully built an API with about 50 queries and over 100 mutations on top of existing Django project (Saleor), but we also learned some hard lessons and had to overcome several shortcomings of the framework along the way.

In this talk, I’d like to share some practical tips to overcome the most common problems that a Django developer might face when building an optimized and maintainable API with Graphene, such as: - using useful abstractions to build queries and mutations faster - optimizing database queries in a graph - structuring a large Graphene project - unified error handling

I’d also like to bring up a few limitations of the framework that we discovered as we were working on the project and then end the talk with the most important benefits that adoption of GraphQL brings to modern web applications development - both for the backend and frontend.

Transcript

1. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE Real world Graphene: lessons learned from building a GraphQL API on top of a large Django project Marcin Gębala
2. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE Who I am ▪ Python developer @ Mirumee Software & development lead @ Saleor ▪ Building and maintaining a GraphQL API for the last two years ▪ Managing the open-source community @ Saleor
3. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE Saleor - GraphQL-first, headless e-commerce github.com/mirumee/saleor
4. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE ▪ Data query language ▪ Open source ▪ Alternative to REST ▪ Minimize amounts of data transferred ▪ Increase developer productivity A bit of history
5. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE ▪ Queries - fetching data ▪ Mutations - modifying data on the server ▪ Subscriptions - real-time data exchange over websockets ▪ All operations are sent as POST requests to a single endpoint /graphql. Operations
6. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE Schema Definition Language Schema is a contract between backend and frontend - it defines data and operations available in API.
7. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE Queries GraphQL syntax Queries allow fetching data from the server.
8. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE Queries JSON response, only requested fields Queries allow fetching data from the server. GraphQL syntax
9. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE Mutations Mutations allow modification of data on the server. Mutation to update a customer
10. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE Mutations Mutations allow modification of data on the server. Server response Mutation to update a customer
11. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE ▪ Bindings for various web frameworks ▪ Building schema using Python classes ▪ Over 4k stars on Github Graphene
12. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE Graphene Lessons learned & recipes
13. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE Project structure ▪ API lives in a single directory: graphql ▪ API modules mirror the structure of Django apps in the project ▪ api.py - imports all API modules and exposes the schema
14. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE Project structure - single module ▪ types.py - definitions of Graphene types, mapping models to types ▪ resolvers.py - resolver functions for queries ▪ mutations.py - definitions of mutation classes ▪ schema.py - gathers all types, queries, and mutations and exposes the part of schema specific for this module
15. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE Graphene Authentication & permissions
16. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE Authentication Using JSON Web Tokens to authenticate users with django-graphql-jwt. Use HTTP Authorization header to pass the token in subsequent requests and authorize them.
17. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE Authentication Using JSON Web Tokens to authenticate users with django-graphql-jwt. Mutation that generates a new token for given credentials Use HTTP Authorization header to pass the token in subsequent requests and authorize them.
18. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE Authentication Using JSON Web Tokens to authenticate users with django-graphql-jwt. Mutation that generates a new token for given credentials Response with the token and user data Use HTTP Authorization header to pass the token in subsequent requests and authorize them.
19. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE Restricting access to queries and mutations Access to particular queries or mutations can be restricted with decorators provided by django-graphql-jwt.
20. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE Restricting access to queries and mutations Access to particular queries or mutations can be restricted with decorators provided by django-graphql-jwt.
21. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE Graphene Unified mutations
22. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE Mutations in Graphene The standard implementation of mutations in Graphene has some limitations: ▪ No input data validation ▪ No standard for returning errors ▪ A lot of boilerplate if the number of mutations is large (in Saleor it is over 150)
23. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE Unified error handling ▪ All mutations return the same “errors” structure ▪ Using Django’s standard ValidationError ▪ Isn’t part of Graphene
24. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE Mutations to create objects CLEAN INPUT DATA CREATE MODEL INSTANCE VALIDATE SAVE INSTANCE Mutations to create new model instances follow the same logic.
25. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE Mutations to create objects CLEAN INPUT DATA CREATE MODEL INSTANCE VALIDATE SAVE INSTANCE Mutations to create new model instances follow the same logic.
26. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE Mutations to create objects CLEAN INPUT DATA CREATE MODEL INSTANCE VALIDATE SAVE INSTANCE Mutations to create new model instances follow the same logic.
27. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE Graphene Database query optimization
28. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE N+1 problem Client can ask for any combination of fields exposed in a type, but some of them are relations in the database. This may result in duplicated database queries. PRODUCT IMAGE * 1
29. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE N+1 problem PRODUCT IMAGE * 1 Client can ask for any combination of fields exposed in a type, but some of them are relations in the database. This may result in duplicated database queries. ▪ products(first:1) - 2 database hits ▪ products(first:20) - 21 database hits
30. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE graphene-django-optimizer Dynamically join related tables with Django’s select_related or prefetch_related functions, based on fields in a query with graphene-django-optimizer.
31. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE Graphene Problems and limitations
32. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE ▪ A fully-fledged API requires multiple additional third-party libraries ▪ Calculating query cost to prevent malicious queries ▪ Serving multiple schemas e.g. private and public API Missing features
33. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE ▪ A lot of open issues and pull requests, uncertain development plans ▪ Lack of good resources, inconsistent docs ▪ Non-pythonic concepts in the source code Maintenance issues
34. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE Real world Graphene Summary
35. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE ▪ GraphQL is a great tool for data exchange between backend and frontend ▪ Graphene allows the gradual addition of GraphQL to your apps ▪ The future of Graphene is uncertain ▪ New libraries are emerging ▪ Despite the limitations, Graphene is still the most developed GraphQL framework for Python My advice: Use Graphene for Django and try new libraries for other Python web frameworks Summary
36. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE Graphene-related: ▪ github.com/graphql-python - Graphene ecosystem ▪ flavors/django-graphql-jwt - JWT authentication ▪ tfoxy/graphene-django-optimizer - database optimization ▪ lmcgartland/graphene-file-upload - file upload Other GraphQL libraries in Python: ▪ mirumee/ariadne - schema-first GraphQL server ▪ dailymotion/tartiflette - asyncio-based GraphQL server ▪ strawberry-graphql/strawberry - GraphQL library based on dataclasses Saleor: ▪ mirumee/saleor Resources
37. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE THANK YOU! QUESTIONS? Marcin Gębala @maarcingebala · maarcingebala · marcin@mirumee.com