Real world Graphene (PyCon Israel 2019)

Graphene is currently the most popular framework for building a GraphQL in Python and it’s also an obvious choice for adding a GraphQL layer to Django applications. Over the course of a year, we successfully built an API with about 50 queries and over 100 mutations on top of existing Django project (Saleor), but we also learned some hard lessons and had to overcome several shortcomings of the framework along the way.

In this talk, I’d like to share some practical tips to overcome the most common problems that a Django developer might face when building an optimized and maintainable API with Graphene, such as: - using useful abstractions to build queries and mutations faster - optimizing database queries in a graph - structuring a large Graphene project - unified error handling

I’d also like to bring up a few limitations of the framework that we discovered as we were working on the project and then end the talk with the most important benefits that adoption of GraphQL brings to modern web applications development - both for the backend and frontend.

Transcript

1. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE Real world Graphene: lessons learned from building a GraphQL API on top of a large Django project Marcin Gębala @maarcingebala
2. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE Who I am ▪ Python developer at Mirumee Software ▪ Development lead at Saleor ▪ Building and maintaining a GraphQL API for the last two years ▪ Managing the open-source community at Saleor @maarcingebala
3. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE Saleor - GraphQL-first e-commerce platform github.com/mirumee/saleor @maarcingebala
4. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE ▪ 50 queries and 150 mutations ▪ Pagination ▪ Filtering, sorting, search ▪ JWT authentication & permissions ▪ Database queries optimization ▪ Public demo: demo.getsaleor.com/graphql Saleor GraphQL API @maarcingebala
5. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE GraphQL Basic concepts @maarcingebala
6. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE ▪ Data query language ▪ Open source ▪ Alternative to REST ▪ Minimize amounts of data transferred ▪ Increase developer productivity A bit of history @maarcingebala
7. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE ▪ Queries - fetching data ▪ Mutations - modifying data on the server ▪ Subscriptions - real-time data exchange over websockets ▪ All operations are sent as POST requests to a single endpoint /graphql Operations @maarcingebala
8. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE ▪ Schema defines the types and operations available in the API ▪ Contract between backend and frontend ▪ Easy to find usages of particular types and track changes in the API Schema @maarcingebala
9. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE Queries GraphQL syntax Queries allow fetching data from the server. @maarcingebala
10. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE Queries JSON response, only requested fields Queries allow fetching data from the server. GraphQL syntax @maarcingebala
11. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE Mutations Mutations allow modification of data on the server. Mutation to update a customer @maarcingebala
12. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE Mutations Mutations allow modification of data on the server. Server response Mutation to update a customer @maarcingebala
13. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE ▪ Bindings for various web frameworks ▪ Building schema using Python classes ▪ Over 4k stars on Github Graphene @maarcingebala
14. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE Graphene Lessons learned & recipes @maarcingebala
15. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE Project structure ▪ API lives in a single directory: graphql ▪ API modules mirror the structure of Django apps in the project ▪ api.py - imports all API modules and exposes the schema @maarcingebala
16. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE Project structure - single module ▪ types.py - definitions of Graphene types, mapping models to types ▪ resolvers.py - resolver functions for queries ▪ mutations.py - definitions of mutation classes ▪ schema.py - gathers all types, queries, and mutations and exposes the part of schema specific for this module @maarcingebala
17. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE Graphene Authentication & permissions @maarcingebala
18. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE Authentication Using JSON Web Tokens to authenticate users with django-graphql-jwt. Use HTTP Authorization header to pass the token in subsequent requests and authorize them. @maarcingebala
19. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE Authentication Using JSON Web Tokens to authenticate users with django-graphql-jwt. Mutation that generates a new token for given credentials Use HTTP Authorization header to pass the token in subsequent requests and authorize them. @maarcingebala
20. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE Authentication Using JSON Web Tokens to authenticate users with django-graphql-jwt. Mutation that generates a new token for given credentials Response with the token and user data Use HTTP Authorization header to pass the token in subsequent requests and authorize them. @maarcingebala
21. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE Restricting access to queries and mutations Access to particular queries or mutations can be restricted with decorators provided by django-graphql-jwt. @maarcingebala
22. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE Restricting access to queries and mutations Access to particular queries or mutations can be restricted with decorators provided by django-graphql-jwt. @maarcingebala
23. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE Graphene Unified errors @maarcingebala
24. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE Mutations in Graphene The standard implementation of mutations in Graphene has some limitations: ▪ Lacks input data validation ▪ No standard for returning errors ▪ A lot of boilerplate if the number of mutations is large @maarcingebala
25. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE Unified error handling ▪ All mutations return the same “errors” structure ▪ Handles validation errors raised by Django’s model validation @maarcingebala
26. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE Graphene Database queries optimization @maarcingebala
27. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE N+1 problem Client can ask for any combination of fields exposed in a type, but some of them are relations in the database. This may result in duplicated database queries. PRODUCT IMAGE * 1 @maarcingebala
28. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE N+1 problem PRODUCT IMAGE * 1 Client can ask for any combination of fields exposed in a type, but some of them are relations in the database. This may result in duplicated database queries. ▪ products(first:1) - 2 database hits ▪ products(first:20) - 21 database hits @maarcingebala
29. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE graphene-django-optimizer Dynamically join related tables with Django’s select_related or prefetch_related functions, based on fields in a query with graphene-django-optimizer. @maarcingebala
30. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE Graphene Problems and limitations @maarcingebala
31. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE ▪ Fully-fledged API requires additional third-party libraries. ▪ No query cost calculation to prevent malicious queries. ▪ Impossible to serve multiple schemas e.g. private and public API. Problems & limitations @maarcingebala
32. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE ▪ Uncertain plans for the future. ▪ Hard to find good learning resources. ▪ Non-pythonic concepts in the source code. Maintenance issues @maarcingebala
33. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE Real world Graphene Summary @maarcingebala
34. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE ▪ GraphQL is a great tool for data exchange between backend and frontend. ▪ Graphene is the most developed GraphQL framework for Python and allows for building powerful APIs. ▪ Future of the framework is uncertain. ▪ New libraries are emerging. Summary @maarcingebala
35. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE Ariadne Ariadne is a Python library for implementing schema-first GraphQL servers that is simple to use and open for extension. ▪ Schema-first approach ▪ Support for asynchronous execution ▪ Inspired by Apollo Server ariadnegraphql.org @maarcingebala
36. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE Graphene-related: ▪ github.com/graphql-python - Graphene ecosystem ▪ flavors/django-graphql-jwt - JWT authentication & permissions ▪ tfoxy/graphene-django-optimizer - database optimization ▪ lmcgartland/graphene-file-upload - file upload Other GraphQL libraries in Python: ▪ mirumee/ariadne - schema-first GraphQL server ▪ dailymotion/tartiflette - asyncio-based GraphQL server ▪ strawberry-graphql/strawberry - GraphQL library based on dataclasses Saleor: ▪ mirumee/saleor - Saleor repository ▪ mirumee/saleor-storefront - PWA storefront Resources @maarcingebala
37. COPYRIGHT © 2009–2019 MIRUMEE SOFTWARE THANK YOU! QUESTIONS? Marcin Gębala @maarcingebala · maarcingebala · marcin@mirumee.com