django CMS @ DUTH - Exploiting Django for a good cause

by Iacopo Spalletti

Slide 1

Slide 1 text

DJANGO CMS@DUTH EXPLOITING DJANGO FOR A GOOD CAUSE

Slide 2

Slide 2 text

HELLO, I AM IACOPO And I keep spamming people about django CMS Founder and CTO @NephilaIt django CMS core developer django CMS installer author

Slide 3

Slide 3 text

DJANGO CMS WHAT'S THAT? A Good Django citizen: handles unstructured content using "pages" integrates with other django apps provides features to other applications glues applications together

Slide 4

Slide 4 text

DJANGO CMS A BIT OF BACKGROUND

Slide 5

Slide 5 text

A BIT OF BACKGROUND Started in 2008 by Divio Derived from django-cms-pages it still possible to port contents from cms-pages (see europython website)

Slide 6

Slide 6 text

A BIT OF BACKGROUND 8 feature releases in 7 years 2.0 ➔ 3.2 Extremely upgradable Django 1.1 / django CMS 2.1 ➔ Django 1.8 / django CMS 3.2

Slide 7

Slide 7 text

DJANGO CMS CONCEPTS

Slide 8

Slide 8 text

DJANGO CMS CONCEPTS Page Title Placeholder Plugin Apphook

Slide 9

Slide 9 text

CONCEPTS PAGE Basic CMS item Tree hierarchy information Associated to a Django template

Slide 10

Slide 10 text

CONCEPTS TITLE Language dependent data: Titles Slug ... Publishing state

Slide 11

Slide 11 text

CONCEPTS PLACEHOLDER Container for plugins Deﬁned in the template

Slide 12

Slide 12 text

CONCEPTS PLUGIN Content Any content managed by the CMS Composed by: Plugin Class Plugin Model

Slide 13

Slide 13 text

CONCEPTS APPHOOK Wrapper for Django applications Allows to integrate Django apps in the CMS tree

Slide 14

Slide 14 text

No content

Slide 15

Slide 15 text

SOME RAW NUMBERS 14400 LOC 3300 admin 2500 models 1000 templatetag 100 view 20000 loc tests 57K downloads/month on PyPi

Slide 16

Slide 16 text

TEMPLATETAGS

Slide 17

Slide 17 text

TEMPLATETAGS django CMS relies on templatetags a lot Frontend Editor Plugin rendering Placeholder discovery ... Two brief examples

Slide 18

Slide 18 text

TEMPLATETAGS Discovering placeholders

{% placeholder "my-placeholder" %}

for node in nodelist: # check if this is a placeholder first if isinstance(node, Placeholder): placeholders.append(node.get_name()) elif isinstance(node, IncludeNode): ... placeholders += _scan_placeholders(...) # handle {% extends ... %} tags elif isinstance(node, ExtendsNode): placeholders += _extend_nodelist(node) # in block nodes we have to scan for super blocks elif isinstance(node, VariableNode) and current_block: ... placeholders += _scan_placeholders(...)

Slide 19

Slide 19 text

TEMPLATETAGS Hijacking the template rendering {% cms_toolbar %} ... def render_tag(self, context, name, nodelist): ... toolbar.populate() ... # render everything below the tag # i.e.: *all* the template rendered_contents = nodelist.render(context) ... toolbar.post_template_populate() ... content = render_to_string( 'cms/toolbar/toolbar.html', context ) return '%s\n%s' % (content, rendered_contents)

Slide 20

Slide 20 text

No content

Slide 21

Slide 21 text

IT'S ALL ABOUT THE ADMIN ~25% of the code is admin related

Slide 22

Slide 22 text

IT'S ALL ABOUT THE ADMIN Frontend editor is admin

Slide 23

Slide 23 text

IT'S ALL ABOUT THE ADMIN Frontend editor is a frontend wrapper around the Django admin No code changes to use it for own applications

Slide 24

Slide 24 text

IT'S ALL ABOUT THE ADMIN Plugins are admin

Slide 25

Slide 25 text

No content

Slide 26

Slide 26 text

IT'S ALL ABOUT THE ADMIN PLUGINS basically a ModelAdmin with a "public" view

Slide 27

Slide 27 text

PLUGINS LIFECYLE Starts at PlaceholderAdmin.add_plugin: def add_plugin(self, request): ... plugin = CMSPlugin(language=language, plugin_type=plugin_type, position=position, placeholder=placeholder) if parent: plugin.position = CMSPlugin.objects.filter(parent=parent).count() plugin.parent_id = parent.pk plugin.save() self.post_add_plugin(request, placeholder, plugin) ... Called via AJAX

Slide 28

Slide 28 text

PLUGINS LIFECYLE The standard change_view is called def edit_plugin(self, request, plugin_id): ... if not instance: # instance doesn't exist, call add view response = plugin_admin.add_view(request) else: # already saved before, call change view # we actually have the instance here, but since i won't override # change_view method, is better if it will be loaded again, so # just pass id to plugin_admin response = plugin_admin.change_view(request, str(plugin_id)) ... return response Now the plugin data are complete

Slide 29

Slide 29 text

LIFECYLE RENDERING How plugin rendering works? Simpliﬁed edition

Slide 30

Slide 30 text

LIFECYLE RENDERING Starts in a templatetag

{% placeholder "my-placeholder" %}

Slide 31

Slide 31 text

LIFECYLE RENDERING Plugin list is retrieved qs = get_cmsplugin_queryset(request) qs = qs.filter(placeholder__in=placeholders, language=lang) plugins = list(qs.order_by('placeholder', 'path')) ... plugins = downcast_plugins(plugins, non_fallback_phs) plugin_groups = dict((key, list(plugins)) for key, plugins in groupby(plugins, attrgetter('placeholder_id'))) for group in plugin_groups: plugin_groups[group] = build_plugin_tree( plugin_groups[group])

Slide 32

Slide 32 text

LIFECYLE RENDERING Template and context are provided by the plugin class context = plugin.render(context, instance, placeholder_slot) ... if plugin.render_plugin: template = plugin._get_render_template(context, instance, placeholder)

Slide 33

Slide 33 text

LIFECYLE RENDERING the plugin is rendered if isinstance(template, six.string_types): content = render_to_string(template, context) elif (isinstance(template, Template) or (hasattr(template, 'template') and hasattr(template, 'render') and isinstance(template.template, Template))): content = template.render(context)

Slide 34

Slide 34 text

LIFECYLE RENDERING If it resembles views, it's not by chance It also uses PLUGIN_CONTEXT_PROCESSORS and PLUGIN_PROCESSORS (i.e. process_response in MIDDLEWARE ) Each step is heavily cached caching attributes on objects using Django cache

Slide 35

Slide 35 text

QUESTIONS?

Slide 36

Slide 36 text

No content

Slide 37

Slide 37 text

ORM Simple models structure

Slide 38

Slide 38 text

ORM MODELS Page and CMSPlugin are organised in tree structures handled via mptt or treebeard

Slide 39

Slide 39 text

MODELS PAGE Website tree structure

Slide 40

Slide 40 text

MODELS PLUGIN

Slide 41

Slide 41 text

The content tree structure

Slide 42

Slide 42 text

MODELS MAIN TOPICS Page publication Plugin polymorphism

Slide 43

Slide 43 text

MODELS PAGE/TITLE PUBLICATION draft and live copy of each page for each language (i.e.: 2 Page and 2xN Title instances exists for any page) each live/draft contains copy of placeholders which contain copy of plugins

Slide 44

Slide 44 text

PAGE / TITLE PUBLICATION

Slide 45

Slide 45 text

PAGE / TITLE PUBLICATION On language (title) publishing: old live title (and plugins) is discarded live title is created as copy of draft draft plugins are copied as live

Slide 46

Slide 46 text

PAGE / TITLE PUBLICATION

Slide 47

Slide 47 text

PAGE / TITLE PUBLICATION

Slide 48

Slide 48 text

PAGE / TITLE PUBLICATION 1. CREATE / GET THE "PUBLIC PAGE" if self.publisher_public_id: # Ensure we have up to date mptt properties public_page = Page.objects.get(pk=self.publisher_public_id) else: public_page = Page(created_by=self.created_by) ... self._copy_attributes(public_page) # we need to set relate this new public copy to its draft page (self public_page.publisher_public = self public_page.publisher_is_draft = False # Ensure that the page is in the right position and save it self._publisher_save_public(public_page) public_page = public_page.reload()

Slide 49

Slide 49 text

PAGE / TITLE PUBLICATION 2. COPY THE PUBLISHED TITLES TO THE PUBLIC PAGE old_titles = dict(target.title_set.filter(language=language).values_list( for title in self.title_set.filter(language=language): old_pk = title.pk # this creates a new instance or recycle the old live title.pk = old_titles.pop(title.language, None) title.page = target title.publisher_is_draft = target.publisher_is_draft title.publisher_public_id = old_pk if published: title.publisher_state = PUBLISHER_STATE_DEFAULT else: title.publisher_state = PUBLISHER_STATE_PENDING title.published = published title._publisher_keep_state = True title.save()

Slide 50

Slide 50 text

PAGE / TITLE PUBLICATION 3. COPY THE CONTENT FOR THE PUBLISHED LANGUAGE for ph in self.get_placeholders(): plugins = ph.get_plugins_list(language) found = False for target_ph in target.placeholders.all(): if target_ph.slot == ph.slot: ph = target_ph found = True break if not found: ph.pk = None # make a new instance ph.save() new_phs.append(ph) if plugins: copy_plugins_to(plugins, ph, no_signals=True)

Slide 51

Slide 51 text

MODELS PLUGIN POLYMORPHISM a.k.a. "Poor man's polymorphism"

Slide 52

Slide 52 text

PLUGIN POLYMORPHISM CMSPlugin is the only "extendable" CMS model Actually you have to class MyCMSPlugin(CMSPlugin): title = models.CharField(max_length=255) ... ORM operations on plugins would be very expensive (e.g.: building the list of plugins assigned to a placeholder) Abstract Base class vs. Multi-table inheritance

Slide 53

Slide 53 text

PLUGIN POLYMORPHISM Polymorphism kicks in Only CMSPlugin model used in complex operations: build list of plugins copy plugins reorder plugins ...

Slide 54

Slide 54 text

PLUGIN POLYMORPHISM DOWNCAST_PLUGINS Before rendering cast from generic to real plugins # make a map of plugin types, needed later for downcasting for plugin in queryset: plugin_types_map[plugin.plugin_type].append(plugin.pk) for plugin_type, pks in plugin_types_map.items(): ... # get all the plugins of type cls.model plugin_qs = cls.get_render_queryset().filter(pk__in=pks) # put them in a map to replace base CMSPlugins with downcasted for instance in plugin_qs: plugin_lookup[instance.pk] = instance ... # make the equivalent list of qs, but with downcasted instances return [plugin_lookup.get(plugin.pk, plugin) for plugin in queryset]

Slide 55

Slide 55 text

PLUGIN POLYMORPHISM COPY_PLUGIN Copying a plugin it's a catch 22 issue tree information created on save saving a plugin blocked by required ﬁelds in concrete model

Slide 56

Slide 56 text

COPY_PLUGIN

Slide 57

Slide 57 text

COPY_PLUGIN QUESTION How to clone the model-speciﬁc data of a model without knowing its structure *and* let mptt/treebeard create tree data?

Slide 58

Slide 58 text

COPY_PLUGIN ANSWER Use the ORM :)

Slide 59

Slide 59 text

COPY_PLUGIN 1. create a generic instance 2. save it (tree data are set) # get the "real" plugin plugin_instance, cls = self.get_plugin_instance() # set up some basic attributes on the new_plugin new_plugin = CMSPlugin() new_plugin.placeholder = target_placeholder parent_cache[self.pk] = new_plugin if self.parent: parent = parent_cache[self.parent_id] parent = CMSPlugin.objects.get(pk=parent.pk) new_plugin.parent_id = parent.pk new_plugin.parent = parent new_plugin.language = target_language new_plugin.plugin_type = self.plugin_type # This triggers the tree setup for the "generic" plugin new_plugin.save()

Slide 60

Slide 60 text

COPY_PLUGIN Swap the plugin model to the concrete type after that if plugin_instance: # get a new instance so references do not get mixed up plugin_instance = plugin_instance.__class__.objects.get( pk=plugin_instance.pk ) plugin_instance.placeholder = target_placeholder # assign the pk of the generic plugin plugin_instance.pk = new_plugin.pk plugin_instance.id = new_plugin.pk # this is the OneToOne field plugin_instance.cmsplugin_ptr = new_plugin plugin_instance.language = target_language # tree information saved before plugin_instance.parent = new_plugin.parent plugin_instance.depth = new_plugin.depth plugin_instance.path = new_plugin.path

Slide 61

Slide 61 text

COPY_PLUGIN Looks weird? A bit Proves the ﬂexibility of the ORM

Slide 62

Slide 62 text

QUESTIONS?

Slide 63

Slide 63 text

No content

Slide 64

Slide 64 text

URLCONF Exploiting the modular nature of Django

Slide 65

Slide 65 text

EXPLOITING THE MODULAR NATURE OF DJANGO URLCONF Enter Apphooks The way to integrate Django apps in the CMS tree

Slide 66

Slide 66 text

APPHOOKS Tiny wrapper around urls.py class MyApp(CMSApp): name = _('My app') urls = ['myapp.urls']

Slide 67

Slide 67 text

APPHOOKS 1. Add an apphook to the page 2. The Apphook urls is injected in the global urlconf 3. magic happens

Slide 68

Slide 68 text

APPHOOKS heavylift in cms/appresolver.py#L215-L245 for page_id in hooked_applications.keys(): resolver = None for lang in hooked_applications[page_id].keys(): (app_ns, inst_ns), current_patterns, app = hooked_applications[pag if not resolver: resolver = AppRegexURLResolver(r'', 'app_resolver', app_name=a resolver.page_id = page_id if app.permissions: _set_permissions(current_patterns, app.exclude_permissions) resolver.url_patterns_dict[lang] = current_patterns app_patterns.append(resolver) APP_RESOLVERS.append(resolver) return app_patterns

Slide 69

Slide 69 text

APPHOOKS 1. get the apphooks for lang in hooked_applications[page_id].keys(): (app_ns, inst_ns), current_patterns, app =\ hooked_applications[page_id][lang]

Slide 70

Slide 70 text

APPHOOKS 2. creates a custom resolver w/ apphook urls 3. attach resolver to the page url (app_ns, inst_ns), current_patterns, app = hooked_applications[page_id][la if not resolver: resolver = AppRegexURLResolver( r'', 'app_resolver', app_name=app_ns, namespace=inst_ns ) resolver.page_id = page_id if app.permissions: _set_permissions(current_patterns, app.exclude_permissions) resolver.url_patterns_dict[lang] = current_patterns

Slide 71

Slide 71 text

APPHOOKS 4. the apphooks resolvers are loaded into urlconf if apphook_pool.get_apphooks(): urlpatterns = get_app_patterns() else: urlpatterns = [] urlpatterns.extend([ url(regexp, details, name='pages-details-by-slug'), url(r'^$', details, {'slug': ''}, name='pages-root'), ]

Slide 72

Slide 72 text

IMPROVEMENT IN 3.1 NAMESPACES CONFIGURATION (via aldryn-apphooks-conﬁg) Detects current namespace Loads a model related to the namespace Attach the model to the current view

Slide 73

Slide 73 text

NAMESPACE CONFIGURATION def get_app_instance(request): ... with override(get_language_from_request(request, check_path=True namespace = resolve(request.path_info).namespace config = app.get_config(namespace) ... def dispatch(self, request, *args, **kwargs): self.namespace, self.config = get_app_instance(request)

Slide 74

Slide 74 text

NAMESPACE CONFIGURATION Example usage class PostListView(AppConfigMixin, ListView): model = Post def get_template_names(self): template_path = self.config.template_prefix \ or 'djangocms_blog' return os.path.join(template_path, self.base_template_name) def get_paginate_by(self, queryset): return self.config.paginate_by or\ get_setting('PAGINATION')

Slide 75

Slide 75 text

IMPROVEMENT IN 3.2 URLCONF RELOAD Into core Uses a middleware 1 DB hit per request Clears the urlconf modules and reload apphook patterns

Slide 76

Slide 76 text

URLCONF RELOAD if 'cms.urls' in sys.modules: reload(sys.modules['cms.urls']) if urlconf is None: urlconf = settings.ROOT_URLCONF if urlconf in sys.modules: reload(sys.modules[urlconf]) clear_app_resolvers() clear_url_caches() get_app_patterns() if new_revision is not None: set_local_revision(new_revision)

Slide 77

Slide 77 text

BONUS INSTANCE AUTO SETUP (via djangocms-apphook-setup) Auto creation of a page and apphook during projects startup 1 hit per project start

Slide 78

Slide 78 text

AUTO SETUP Sample Apphook with namespace conﬁguration auto setup conﬁguration class BlogApp(AutoCMSAppMixin, CMSConfigApp): name = _('Blog') urls = ['djangocms_blog.urls'] app_name = 'djangocms_blog' app_config = BlogConfig auto_setup = { ... 'page title': get_setting('AUTO_BLOG_TITLE'), 'namespace': get_setting('AUTO_NAMESPACE'), 'config_fields': {...}, 'config_translated_fields': {...}, } apphook_pool.register(BlogApp) BlogApp.setup()