students stipends to write code for open source projects • One of Google’s ways to give back to Open Source • A great occasion for you to give back to open source! • May 19 to August 18 • Nearly 200 Open Source organisations • Each organisation has 1 or more proposed projects (you can also propose others) • 5,000 USD for the project Program details
each model • Allows Django to introspect a model’s internals • Makes a lot of Django’s model magic possible • … so basically you don’t know about it, but you may have used it What is the Meta API A very simple definition
model • Field names • Field instances • Model relations • Field attributes What is inside the Meta object What is the Meta API >>> User._meta.fields (<django.db.models.fields.AutoField: id>, <django.db.models.fields.CharField: password>, <django.db.models.fields.DateTimeField: last_login>, ... ... )
has a ForeignKey to Brand 1 Brand has a RelatedObject to Item as a consequence of the relation 2 Related Objects Related objects are objects created as a consequence of a relation of another model with the current model. Fields Any field defined on the model with or without a relation.
related_objects related_m2m Excludes RO coming from M2M relations Only RO coming from M2M relations All Model fields fields many_to_many Excludes fields that are M2M Only fields that are M2M The decision to split these concepts into multiple properties is only an implementation detail
• Generates a map between models and connections • Efficient, computed once for everyone and cached • Still really expensive on first lookup: For every model in every field in every app The new Meta API Related objects graph
of Django, therefore it must provide excellent memory management. Immutable data structures allocate exactly the required space, without over-allocating. • Reusability By returning a reference to an immutable data structure, we guarantee that the end-user cannot manipulate the array, and therefore we can safely return the same reference. • Less bug-prone Personal experience here! API consumers will often retain the array as their own and manipulate the contents. Immutability Why immutability
return types are immutable, no copies are returned • All return types are cached • When possible, we use data structures that derive from set and tuple Immutability in the Meta API Immutability How does this impact how Django consumes the API? • Iteration over multiple API calls is done using itertools.chain() • Use generators everywhere, when filtering API results
list • Use generator comprehension to map or filter API results Downfalls of generator expressions: no indexing or multiple iteration. Currently this happens very little.. Immutability Immutability in the Meta API
to previous and current API • Performance increased compared to previous API • 465 commits in my second PR, as of today! • A fully working, refactored API, for what we have today in Django • ..But, not what we may want to have tomorrow The new Meta API What we have up till now 10% Performance increase DjangoBench, median of 1000 runs for each test
column field.is_relation Field has relation with another model field.one_to_many Cardinality 1-N field.many_to_one Cardinality N-1 field.many_to_many Cardinality N-N field.one_to_one Cardinality 1-1 The future Meta API Boolean flags
field’s functionality (ex. GenericForeignKey) field.model The model that contains the field field.referred_model The model that a field points to (in the case the field has a relation) The future Meta API Data flags
FIELDS = (f for f in A._meta.get_fields() if not f.hidden and f.is_relation and f.referred_model == B) # Fetch all fields to show on a form for A (including Fks) FIELDS = (f for f in A._meta.get_fields() if not f.hidden and f.editable and (not f.has_relation or f.one_to_many)) # Fetch all fields that have a connected db column FIELDS = (f for f in A._meta.get_fields() if f.concrete) The future Meta API Querying with get_fields() and field flags
include_hidden, include_proxy, include_concrete): Naming things Making the API even more sparse • Flags are far better: more pythonic and less imports • This matrix can describe exactly what we have now • This matrix may not describe what we want in the future • Field types and options are too sparse to be api parameters.
2 main field distinctions • Only separates the main 2 distinction points • The rest of the filtering is done outside the API • Far simpler and easier to maintain • We are not there yet, as this distinction may not exactly be what we want in the future (future ForeignKeys, future Virtual Fields)
far: Russell Keith Magee, Collin Anderson, Tim Graham, Loic Bistuer, Anssi, and many more Without these people, I wouldn’t be speaking here: Mark Tamlyn, Dutch Django association, Ola Sitarska, and many more! Community Daniel Pyrathon @pirosb3 Naming things A huge thanks