Developing custom Queryset methods

Chibisov Gennady

Разработка и тестирование кастомных QuerySet-методов

… а также методов менеджеров и инстансов моделей на основе предикатов, не нарушая принципов DRY.

Transcript

1. Developing  custom  QuerySet  
   methods  
   Chibisov  Gennady  
   

   View Slide

2. https://github.com/chibisov/events-site-example
   !
   

   View Slide

3. Model  methods  
   1. Instance  methods  
   2. Class  methods  
   3. QuerySet  methods  
   4. Manager  methods  
   

   View Slide

4. Instance  methods  
   class Event(models.Model): !
   name = models.CharField(max_length=255)!
   slug = models.SlugField(unique=True)!
   date_start = models.DateTimeField()!
   !
   def __unicode__(self):!
   return self.name !
   !
   @property!
   def is_published(self):!
   # uses current instance data!
   return (self.date_start <=!
   datetime.datetime.now())!
   !
   !
   >>> yac2012 = Event.objects.get(slug='yac2012')!
   >>> yac2012.__unicode__()!
   'Yet another Conference 2012'!
   >>> yac2012.is_published!
   True!
    
   

   View Slide

5. Class  methods  
   class Event(models.Model): !
   name = models.CharField(max_length=255)!
   slug = models.SlugField(unique=True)!
   date_start = models.DateTimeField()!
   !
   @classmethod!
   def get_fields_for_csv(cls):!
   return ['name', 'slug']!
   !
   !
   >>> Event.get_fields_for_csv()!
   ['name', 'slug']!
    
   

   View Slide

6. QuerySet  methods  
   Event.objects.filter(id__lte=10).delete()!
   

   View Slide

7. QuerySet  methods  
   Event.objects.all().get_or_create(…)!
   Event.objects.all().create(…)!
   Event.objects.all().bulk_create(…)  
   

   View Slide

8. Manager  methods  
   class RegionManager(models.Manager):!
   def sync_with_geobase(self):!
   for region in regions_from_geobase():!
   try:!
   region_instance = self.model.objects.get(geobase_id=region.id)!
   # can modify any instance!
   region_instance.name_ru = region.name!
   region_instance.name_en = region.ename!
   region_instance.save()!
   except self.model.DoesNotExist:!
   # can create new instances!
   region_instance = self.model.objects.create(!
   geobase_id=region, !
   name_ru=region.name, !
   name_en=region.ename!
   )!
   # can remove instance!
   if region.is_deprecated:!
   region_instance.delete()!
   !
   !
   >>> Region.objects.sync_with_geobase()!
   

   View Slide

9. Manager  methods  
   # app/management/commands/sync_with_geobase.py!
   class Command(BaseCommand):!
   help = u"""Syncs regions with geobase"""!
   !
   def handle(self, *args, **options):!
   Region.objects.sync_with_geobase()!
   !
   !
   $ ./manage.py sync_with_geobase!
   !
   

   View Slide

10. This  talk  
    1. Instance  methods  
    2. Class  methods  
    3. QuerySet  methods  
    4. Manager  methods  
    

    View Slide

11. What  for?  
    

    View Slide

12. What  for?  
    1. Readability  
    

    View Slide

13. Readability  L  
    events = Events.objects.all()!
    # filter for current site type!
    if settings.IS_INTERNAL_SITE:!
    events = events.filter(!
    Q(is_published_external=True) | !
    Q(is_published_internal=True)!
    )!
    else:!
    events = events.filter(!
    is_published_for_external=True!
    )!
    

    View Slide

14. Readability  J  
    events = Events.objects.filter_for_current_site_type()!
    

    View Slide

15. Readability  L  
    yac2012 = Events.objects.get(slug='yac2012')!
    !
    # is for current site type?!
    if settings.IS_INTERNAL_SITE:!
    is_for = (yac2012.is_published_for_internal !
    and yac2012.is_published_for_external)!
    else:!
    is_for = yac2012.is_published_for_external!
    

    View Slide

16. Readability  J  
    yac2012.is_for_current_site_type!
    

    View Slide

17. What  for?  
    1. Readability  
    2. Reusability  
    

    View Slide

18. Reusability  J  
    class EventDetailView(DetailView): !
    queryset = (Event.objects!
    .filter_for_current_site_type())!
    !
    !
    class EventListView(ListView):!
    queryset = (Event.objects!
    .filter_for_current_site_type())!
    

    View Slide

19. First  manager  
    # apps/events/managers.py !
    from datetime.datetime import now!
    from django.db.models.query import QuerySet!
    from django.db import models!
    !
    !
    class EventQuerySet(QuerySet):!
    def filter_by_published(self):!
    return self.filter(date_start__lte=now())!
    !
    !
    class EventManager(models.Manager):!
    def get_query_set(self):!
    return EventQuerySet(self.model, !
    using=self._db)!
    

    View Slide

20. First  manager  
    # apps/events/models.py!
    from django.db import models!
    from apps.events.managers import EventManager!
    !
    !
    class Event(models.Model):!
    name = models.CharField(max_length=255)!
    slug = models.SlugField(unique=True)!
    date_start = models.DateTimeField()!
    !
    objects = EventManager()!
    

    View Slide

21. First  manager  
    >>> Event.objects.all().filter_by_published()!
    !
    [, !
    ]!
    

    View Slide

22. Oops…  
    >>> Event.objects.filter_by_published()!
    !
    Traceback (most recent call last):!
    File "", line 1, in !
    AttributeError: 'EventManager' object has no
    attribute 'filter_by_published'!
    

    View Slide

23. Add  proxy  method  
    class EventManager(models.Manager):!
    def get_query_set(self):!
    return EventQuerySet(self.model, !
    using=self._db)!
    !
    def filter_by_published(self):!
    return (self.get_query_set()!
    .filter_by_published())!
    

    View Slide

24. Or  try  to  proxy  all  not  found  
    class EventManager(models.Manager):!
    def get_query_set(self):!
    return EventQuerySet(self.model, !
    using=self._db)!
    !
    def __getattr__(self, name):!
    if name.startswith('_'):!
    raise AttributeError(name)!
    else:!
    return getattr(self.get_query_set(), !
    name)!
    

    View Slide

25. OK  
    >>> Event.objects.filter_by_published()!
    !
    [, !
    ]!
    

    View Slide

26. More  
    docs.djangoproject.com/en/dev/topics/db/managers/!
    

    View Slide

27. apps.events.models  
    class EventType(models.Model):!
    """!
    Example: Yet Another Conference!
    """!
    name = models.CharField(max_length=255)!
    slug = models.SlugField(unique=True)!
    is_moderated = models.BooleanField(default=False)!
    

    View Slide

28. /events/  
    •  Yet  another  Conference  
    •  I.Subbotnik  
    •  ECIR  
    

    View Slide

29. apps.events.models  
    class Event(models.Model):!
    """!
    Example: Yac 2012!
    """!
    name = models.CharField(max_length=255)!
    slug = models.SlugField(unique=True)!
    type = models.ForeignKey(EventType)!
    date_start = models.DateTimeField()!
    

    View Slide

30. /events/yac/  
    •  Yac  2010  
    •  Yac  2011  
    •  Yac  2012  
    

    View Slide

31. /events/yac/2012/  
    

    View Slide

32. /events/  
    class EventTypeListView(ListView):!
    queryset = EventType.objects.all()!
     
    

    View Slide

33. /events/  
    •  Yet  another  Conference  
    •  I.Subbotnik  
    •  ECIR  (not  moderated)  
    

    View Slide

34. EventType  QuerySet  method  
    class EventTypeQuerySet(QuerySet):!
    def filter_by_published(self):!
    return self.filter(is_moderated=True)!
    !
    

    View Slide

35. /events/  
    class EventTypeListView(ListView):!
    queryset = (EventType.objects!
    .filter_by_published())!
     
    

    View Slide

36. /events/  
    •  Yet  another  Conference  
    •  I.Subbotnik  
    

    View Slide

37. /events/yac/  
    •  Yac  2010  
    •  Yac  2011  
    •  Yac  2012  (too  early)  
    

    View Slide

38. Event  QuerySet  method  
    import datetime!
    from datetime.datetime import now!
    !
    class EventQuerySet(QuerySet):!
    def filter_by_it_time_to_publish(self):!
    d = now() + datetime.timedelta(days=2)!
    return self.filter(date_start__lte=d)!
    

    View Slide

39. /events/yac/  
    class EventTypeDetailView(DetailView):!
    queryset = (EventType.objects!
    .filter_by_published())!
     
    

    View Slide

40. /events/yac/  
    # /templates/events/eventtype_detail.html!
    !
    {% for e in object.event_set!
    .filter_by_it_time_to_publish %}!
    {{ e }}!
    {% endfor %}!
    

    View Slide

41. /events/yac/  
    •  Yac  2010  
    •  Yac  2011  
    

    View Slide

42. /events/yac/2012/  
    

    View Slide

43. /events/yac/2012/  
    Show  if  
     
    

    View Slide

44. /events/yac/2012/  
    Show  if  
    •  It  is  Tme  to  publish  
    

    View Slide

45. /events/yac/2012/  
    Show  if  
    •  It  is  Tme  to  publish  
    •  Yac  EventType  is  published  
    

    View Slide

46. /events/yac/2012/  
    else  
    404  
    

    View Slide

47. /events/yac/2012/  
    class EventDetailView(DetailView):!
    queryset = (Event.objects!
    .filter_by_it_time_to_publish())!
    !
    

    View Slide

48. /events/yac/2012/  
    class EventDetailView(DetailView):!
    queryset = (Event.objects!
    .filter_by_it_time_to_publish())!
    !
    def get_queryset(self):!
    q = super(EventDetailView, !
    self).get_queryset()!
    return q.filter(type__is_moderated=True)!
    

    View Slide

49. L  
     
    EventType.objects.filter_by_published()  
    not  reused  
    

    View Slide

50. /events/yac/2012/  
    class EventDetailView(DetailView):!
    queryset = (Event.objects!
    .filter_by_it_time_to_publish())!
    !
    def get_queryset(self):!
    q = super(EventDetailView, !
    self).get_queryset()!
    

    View Slide

51. /events/yac/2012/  
    class EventDetailView(DetailView):!
    queryset = (Event.objects!
    .filter_by_it_time_to_publish())!
    !
    def get_queryset(self):!
    q = super(EventDetailView, !
    self).get_queryset()!
    pks = (EventType.objects!
    .filter_by_published()!
    .values_list('pk', flat=True))!
    

    View Slide

52. /events/yac/2012/  
    class EventDetailView(DetailView):!
    queryset = (Event.objects!
    .filter_by_it_time_to_publish())!
    !
    def get_queryset(self):!
    q = super(EventDetailView, !
    self).get_queryset()!
    pks = (EventType.objects!
    .filter_by_published()!
    .values_list('pk', flat=True))!
    return q.filter(type__pk__in=pks)!
    

    View Slide

53. L  
     
    2  DB  queries  
    Ugly  
    

    View Slide

54. /events/yac/2012/  
    class EventDetailView(DetailView):!
    queryset = (Event.objects!
    .filter_by_published())!
    

    View Slide

55. Pre\y  
    1  query  
    Reuse  filter_by_published  
    

    View Slide

56. HOW?  
    

    View Slide

57. Q  object  
    

    View Slide

58. class EventTypeQuerySet(QuerySet):!
    def filter_by_published(self):!
    return self.filter(Q(is_moderated=True))!
    !
    !
    class EventQuerySet(QuerySet):!
    ...!
    def filter_by_with_published_type(self):!
    return self.filter(Q(type__is_moderated=True))!
    

    View Slide

59. class EventTypeQuerySet(QuerySet):!
    def filter_by_published(self):!
    return self.filter(Q(is_moderated=True))!
    !
    !
    class EventQuerySet(QuerySet):!
    ...!
    def filter_by_with_published_type(self):!
    return self.filter(Q(type__is_moderated=True))!
    Diff  
    

    View Slide

60. class EventTypeConditions(object):!
    @classmethod!
    def is_published(cls):!
    return Q(is_moderated=True)!
    Move  Q  to  funcTon  
    

    View Slide

61. class EventTypeConditions(object):!
    @classmethod!
    def is_published(cls, prefix=None): !
    return Q(is_moderated=True)!
    Prefix  as  param  
    

    View Slide

62. class EventTypeConditions(object):!
    @classmethod!
    def is_published(cls, prefix=None):!
    if prefix:!
    field = '%s__is_moderated' % prefix!
    else:!
    field = 'is_moderated'!
    !
    return Q(is_moderated=True)!
    Field  name  with  prefix  
    

    View Slide

63. class EventTypeConditions(object):!
    @classmethod!
    def is_published(cls, prefix=None):!
    if prefix:!
    field = '%s__is_moderated' % prefix!
    else:!
    field = 'is_moderated'!
    !
    return Q(**{field: True})!
    Use  prefixed  field  
    

    View Slide

64. >>> EventTypeConditions.is_published()!
    

    View Slide

65. >>> EventTypeConditions.is_published()!
    Q(is_moderated=True)!
    

    View Slide

66. >>> EventTypeConditions.is_published()!
    Q(is_moderated=True)!
    !
    >>> EventTypeConditions.is_published(!
    ... prefix='type'!
    ... )!
    

    View Slide

67. >>> EventTypeConditions.is_published()!
    Q(is_moderated=True)!
    !
    >>> EventTypeConditions.is_published(!
    ... prefix='type'!
    ... )!
    Q(type__is_moderated=True)!
    

    View Slide

68. class EventTypeQuerySet(QuerySet):!
    def filter_by_published(self):!
    return self.filter(!
    EventTypeConditions.is_published()!
    )!
    Use  it  for  EventType  
    

    View Slide

69. May  I?  
    class EventQuerySet(QuerySet):!
    ...!
    def filter_by_with_published_type(self):!
    return self.filter(!
    EventTypeConditions.is_published(!
    prefix='type’!
    )!
    )!
    

    View Slide

70. NO  
    

    View Slide

71. NO  
    EventTypeCondiTons  is  
    for  EventType  
    

    View Slide

72. NO  
    EventTypeCondiTons  is  
    for  EventType  
    Create  
    EventCondiTons  for  Event  
    

    View Slide

73. class EventConditions(object):!
    @classmethod!
    def is_with_published_type(cls):!
    event_prefix = 'type' !
    return EventTypeConditions.is_published(!
    prefix=event_prefix!
    ) !
    Create  Event  condiTons  
    

    View Slide

74. class EventConditions(object):!
    @classmethod!
    def is_with_published_type(cls, prefix=None): !
    event_prefix = 'type' !
    return EventTypeConditions.is_published(!
    prefix=event_prefix!
    )!
    Prefix  as  param  
    

    View Slide

75. class EventConditions(object):!
    @classmethod!
    def is_with_published_type(cls, prefix=None):!
    event_prefix = 'type' !
    if prefix:!
    prefix = '%s__%s' % prefix, event_prefix!
    else:!
    prefix = event_prefix!
    !
    return EventTypeConditions.is_published(!
    prefix=event_prefix!
    )!
    Add  prefix  
    

    View Slide

76. class EventConditions(object):!
    @classmethod!
    def is_with_published_type(cls, prefix=None):
    event_prefix = 'type'!
    if prefix:!
    prefix = '%s__%s' % prefix, event_prefix!
    else:!
    prefix = event_prefix!
    !
    return EventTypeConditions.is_published(!
    prefix=prefix!
    )!
    Use  prefix  
    

    View Slide

77. >>> EventConditions.is_with_published_type()!
    

    View Slide

78. >>> EventConditions.is_with_published_type()!
    Q(type__is_moderated=True)!
    !
    

    View Slide

79. >>> EventConditions.is_with_published_type()!
    Q(type__is_moderated=True)!
    !
    >>> EventConditions.is_with_published_type(!
    ... prefix=’event’!
    ... )!
    

    View Slide

80. >>> EventConditions.is_with_published_type()!
    Q(type__is_moderated=True)!
    !
    >>> EventConditions.is_with_published_type(!
    ... prefix=’event’!
    ... )!
    Q(event__type__is_moderated=True)!
    

    View Slide

81. Use  it  for  Event  
    class EventQuerySet(QuerySet):!
    ...!
    def filter_by_with_published_type(self):!
    return self.filter(!
    EventConditions.is_with_published_type()!
    )!
    

    View Slide

82. Lets  refactor  
    class EventQuerySet(QuerySet):!
    ...!
    def filter_by_it_time_to_publish(self):!
    d = now() + datetime.timedelta(days=2)!
    return self.filter(date_start__lte=d)!
    

    View Slide

83. class EventConditions(object):!
    @classmethod!
    def is_with_published_type(cls, prefix=None):!
    ...!
    !
    @classmethod!
    def is_it_time_to_publish(cls, prefix=None): !
    if prefix:!
    field = '%s__date_start__lte' % prefix!
    else:!
    field = 'date_start__lte'!
    !
    d = now() + datetime.timedelta(days=2) !
    return Q(**{field: d})!
    

    View Slide

84. Aaer  refactor  
    class EventQuerySet(QuerySet):!
    ...!
    def filter_by_it_time_to_publish(self):!
    return self.filter(!
    EventConditions.is_it_time_to_publish()!
    )!
    

    View Slide

85. Combining  methods  
    

    View Slide

86. class EventConditions(object):!
    @classmethod!
    def is_with_published_type(cls, prefix=None):!
    ...!
    !
    @classmethod!
    def is_it_time_to_publish(cls, prefix=None): !
    ...!
    

    View Slide

87. class EventConditions(object):!
    @classmethod!
    def is_with_published_type(cls, prefix=None):!
    ...!
    !
    @classmethod!
    def is_it_time_to_publish(cls, prefix=None): !
    ...!
    !
    @classmethod!
    def is_published(cls, prefix=None):!
    return (!
    cls.is_with_published_type(prefix) & !
    cls.is_it_time_to_publish(prefix)!
    )!
    !
    

    View Slide

88. >>> EventConditions.is_published()!
    !
    

    View Slide

89. >>> EventConditions.is_published()!
    Q(date_start__lte=datetime(23,11,2012)) &!
    Q(type__is_moderated=True)!
    !
    

    View Slide

90. >>> EventConditions.is_published()!
    Q(date_start__lte=datetime(23,11,2012)) &!
    Q(type__is_moderated=True)!
    !
    >>> EventConditions.is_published(!
    ... prefix=’event’!
    ... )!
    

    View Slide

91. >>> EventConditions.is_published()!
    Q(date_start__lte=datetime(23,11,2012)) &!
    Q(type__is_moderated=True)!
    !
    >>> EventConditions.is_published(!
    ... prefix=’event’!
    ... )!
    Q(event__date_start__lte=datetime(23,11,2012))&!
    Q(event__type__is_moderated=True)!
    

    View Slide

92. Model  instance  methods  
    

    View Slide

93.  
     
    How  to  reuse  condiTons?  
    

    View Slide

94. How  to  reuse  condiTons?  
    class  Event(models.Model):  
           ...  
           @property  
           def  is_with_published_type(self):  
                   return  EventConditions.is_with_published_type(model_instance=self)  
     
           @property  
           def  is_it_time_to_publish(self):  
                   return  EventConditions.is_it_time_to_publish(model_instance=self)  
     
           @property  
           def  is_published(self):  
                   return  EventConditions.is_published(model_instance=self)!
    

    View Slide

95. True  and  False  
    =  
    True  &  False  
     
     
    How  to  reuse  condiTons?  
    

    View Slide

96. True  or  False  
    =  
    True  |  False  
     
     
    How  to  reuse  condiTons?  
    

    View Slide

97. More  
    https://github.com/chibisov/events-site-example
    !
    /apps/accounts/models.py  
    /apps/accounts/managers.py  
     
    /apps/events/models.py  
    /apps/events/managers.py  
     
    

    View Slide

98. How  to  test  
    

    View Slide

99. Use  makers  
    •  make_published  
    •  make_not_published  
    

    View Slide

100. Use  makers  
     class TestEventIsPublished(TestConditionBase):!
     def test_1(self):!
     makers.event.make_with_published_type(self.instance)!
     makers.event.make_it_time_to_publish(self.instance)!
     !
     self.assertConditionTrue(msg=msg)!
     !
     def test_2(self):!
     makers.event.make_with_published_type(self.instance)!
     makers.event.make_not_it_time_to_publish(self.instance)!
     !
     self.assertConditionFalse(msg=msg)!
     !
     def test_3(self):!
     makers.event.make_it_time_to_publish(self.instance)!
     makers.event.make_not_with_published_type(self.instance)!
     !
     self.assertConditionFalse(msg=msg)!
     

     View Slide

101. More  
     https://github.com/chibisov/events-site-example
     !
     /apps/accounts/makers/  
     /apps/accounts/tests/condiTons/  
      
     /apps/events/makers/  
     /apps/events/tests/condiTons/  
      
     

     View Slide

102. class DistributionSendedConditions(object):!
     @classmethod!
     def is_ready_to_start_with_in_progress_status(cls, prefix=None):!
     k = {!
     'prefix': prefix!
     }!
     !
     return (!
     (cls.is_with_status(status='in_progress', **k)) &!
     ((cls.is_last_message_sended_more_than_5_minutes_ago(**k)) |!
     (cls.is_not_has_messages(**k) & !
     cls.is_started_sending_more_than_5_minutes_ago(**k)))!
     )!
     More  complex  condiTons  
     

     View Slide

103. Спасибо  
     

     View Slide