Share
Embed
django CMS @ DUTH - Exploiting Django for a good cause
Given at Django: Under the Hood 2015 05 November 2015
Video: https://opbeat.com/community/posts/django-cms-and-orm-by-iacopo-spalletti/
Iacopo Spalletti
November 05, 2015
Transcript
81 slides
-
HELLO, I AM IACOPO And I keep spamming people about
django CMS Founder and CTO @NephilaIt django CMS core developer django CMS installer author -
DJANGO CMS WHAT'S THAT? A Good Django citizen: handles unstructured
content using "pages" integrates with other django apps provides features to other applications glues applications together -
A BIT OF BACKGROUND Started in 2008 by Divio Derived
from django-cms-pages it still possible to port contents from cms-pages (see europython website) -
A BIT OF BACKGROUND 8 feature releases in 7 years
2.0 ➔ 3.2 Extremely upgradable Django 1.1 / django CMS 2.1 ➔ Django 1.8 / django CMS 3.2 -
CONCEPTS PAGE Basic CMS item Tree hierarchy information Associated to
a Django template -
CONCEPTS PLUGIN Content Any content managed by the CMS Composed
by: Plugin Class Plugin Model -
CONCEPTS APPHOOK Wrapper for Django applications Allows to integrate Django
apps in the CMS tree -
SOME RAW NUMBERS 14400 LOC 3300 admin 2500 models 1000
templatetag 100 view 20000 loc tests 57K downloads/month on PyPi -
TEMPLATETAGS django CMS relies on templatetags a lot Frontend Editor
Plugin rendering Placeholder discovery ... Two brief examples -
TEMPLATETAGS Discovering placeholders <div class="content">{% placeholder "my-placeholder" %}</div> for node
in nodelist: # check if this is a placeholder first if isinstance(node, Placeholder): placeholders.append(node.get_name()) elif isinstance(node, IncludeNode): ... placeholders += _scan_placeholders(...) # handle {% extends ... %} tags elif isinstance(node, ExtendsNode): placeholders += _extend_nodelist(node) # in block nodes we have to scan for super blocks elif isinstance(node, VariableNode) and current_block: ... placeholders += _scan_placeholders(...) -
TEMPLATETAGS Hijacking the template rendering {% cms_toolbar %} ... def
render_tag(self, context, name, nodelist): ... toolbar.populate() ... # render everything below the tag # i.e.: *all* the template rendered_contents = nodelist.render(context) ... toolbar.post_template_populate() ... content = render_to_string( 'cms/toolbar/toolbar.html', context ) return '%s\n%s' % (content, rendered_contents) -
IT'S ALL ABOUT THE ADMIN ~25% of the code is
admin related -
IT'S ALL ABOUT THE ADMIN Frontend editor is a frontend
wrapper around the Django admin No code changes to use it for own applications -
IT'S ALL ABOUT THE ADMIN PLUGINS basically a ModelAdmin with
a "public" view -
PLUGINS LIFECYLE Starts at PlaceholderAdmin.add_plugin: def add_plugin(self, request): ... plugin
= CMSPlugin(language=language, plugin_type=plugin_type, position=position, placeholder=placeholder) if parent: plugin.position = CMSPlugin.objects.filter(parent=parent).count() plugin.parent_id = parent.pk plugin.save() self.post_add_plugin(request, placeholder, plugin) ... Called via AJAX -
PLUGINS LIFECYLE The standard change_view is called def edit_plugin(self, request,
plugin_id): ... if not instance: # instance doesn't exist, call add view response = plugin_admin.add_view(request) else: # already saved before, call change view # we actually have the instance here, but since i won't override # change_view method, is better if it will be loaded again, so # just pass id to plugin_admin response = plugin_admin.change_view(request, str(plugin_id)) ... return response Now the plugin data are complete -
LIFECYLE RENDERING Plugin list is retrieved qs = get_cmsplugin_queryset(request) qs
= qs.filter(placeholder__in=placeholders, language=lang) plugins = list(qs.order_by('placeholder', 'path')) ... plugins = downcast_plugins(plugins, non_fallback_phs) plugin_groups = dict((key, list(plugins)) for key, plugins in groupby(plugins, attrgetter('placeholder_id'))) for group in plugin_groups: plugin_groups[group] = build_plugin_tree( plugin_groups[group]) -
LIFECYLE RENDERING Template and context are provided by the plugin
class context = plugin.render(context, instance, placeholder_slot) ... if plugin.render_plugin: template = plugin._get_render_template(context, instance, placeholder) -
LIFECYLE RENDERING the plugin is rendered if isinstance(template, six.string_types): content
= render_to_string(template, context) elif (isinstance(template, Template) or (hasattr(template, 'template') and hasattr(template, 'render') and isinstance(template.template, Template))): content = template.render(context) -
LIFECYLE RENDERING If it resembles views, it's not by chance
It also uses PLUGIN_CONTEXT_PROCESSORS and PLUGIN_PROCESSORS (i.e. process_response in MIDDLEWARE ) Each step is heavily cached caching attributes on objects using Django cache -
ORM MODELS Page and CMSPlugin are organised in tree structures
handled via mptt or treebeard -
MODELS PAGE/TITLE PUBLICATION draft and live copy of each page
for each language (i.e.: 2 Page and 2xN Title instances exists for any page) each live/draft contains copy of placeholders which contain copy of plugins -
PAGE / TITLE PUBLICATION On language (title) publishing: old live
title (and plugins) is discarded live title is created as copy of draft draft plugins are copied as live -
PAGE / TITLE PUBLICATION 1. CREATE / GET THE "PUBLIC
PAGE" if self.publisher_public_id: # Ensure we have up to date mptt properties public_page = Page.objects.get(pk=self.publisher_public_id) else: public_page = Page(created_by=self.created_by) ... self._copy_attributes(public_page) # we need to set relate this new public copy to its draft page (self public_page.publisher_public = self public_page.publisher_is_draft = False # Ensure that the page is in the right position and save it self._publisher_save_public(public_page) public_page = public_page.reload() -
PAGE / TITLE PUBLICATION 2. COPY THE PUBLISHED TITLES TO
THE PUBLIC PAGE old_titles = dict(target.title_set.filter(language=language).values_list( for title in self.title_set.filter(language=language): old_pk = title.pk # this creates a new instance or recycle the old live title.pk = old_titles.pop(title.language, None) title.page = target title.publisher_is_draft = target.publisher_is_draft title.publisher_public_id = old_pk if published: title.publisher_state = PUBLISHER_STATE_DEFAULT else: title.publisher_state = PUBLISHER_STATE_PENDING title.published = published title._publisher_keep_state = True title.save() -
PAGE / TITLE PUBLICATION 3. COPY THE CONTENT FOR THE
PUBLISHED LANGUAGE for ph in self.get_placeholders(): plugins = ph.get_plugins_list(language) found = False for target_ph in target.placeholders.all(): if target_ph.slot == ph.slot: ph = target_ph found = True break if not found: ph.pk = None # make a new instance ph.save() new_phs.append(ph) if plugins: copy_plugins_to(plugins, ph, no_signals=True) -
PLUGIN POLYMORPHISM CMSPlugin is the only "extendable" CMS model Actually
you have to class MyCMSPlugin(CMSPlugin): title = models.CharField(max_length=255) ... ORM operations on plugins would be very expensive (e.g.: building the list of plugins assigned to a placeholder) Abstract Base class vs. Multi-table inheritance -
PLUGIN POLYMORPHISM Polymorphism kicks in Only CMSPlugin model used in
complex operations: build list of plugins copy plugins reorder plugins ... -
PLUGIN POLYMORPHISM DOWNCAST_PLUGINS Before rendering cast from generic to real
plugins # make a map of plugin types, needed later for downcasting for plugin in queryset: plugin_types_map[plugin.plugin_type].append(plugin.pk) for plugin_type, pks in plugin_types_map.items(): ... # get all the plugins of type cls.model plugin_qs = cls.get_render_queryset().filter(pk__in=pks) # put them in a map to replace base CMSPlugins with downcasted for instance in plugin_qs: plugin_lookup[instance.pk] = instance ... # make the equivalent list of qs, but with downcasted instances return [plugin_lookup.get(plugin.pk, plugin) for plugin in queryset] -
PLUGIN POLYMORPHISM COPY_PLUGIN Copying a plugin it's a catch 22
issue tree information created on save saving a plugin blocked by required fields in concrete model -
COPY_PLUGIN QUESTION How to clone the model-specific data of a
model without knowing its structure *and* let mptt/treebeard create tree data? -
COPY_PLUGIN 1. create a generic instance 2. save it (tree
data are set) # get the "real" plugin plugin_instance, cls = self.get_plugin_instance() # set up some basic attributes on the new_plugin new_plugin = CMSPlugin() new_plugin.placeholder = target_placeholder parent_cache[self.pk] = new_plugin if self.parent: parent = parent_cache[self.parent_id] parent = CMSPlugin.objects.get(pk=parent.pk) new_plugin.parent_id = parent.pk new_plugin.parent = parent new_plugin.language = target_language new_plugin.plugin_type = self.plugin_type # This triggers the tree setup for the "generic" plugin new_plugin.save() -
COPY_PLUGIN Swap the plugin model to the concrete type after
that if plugin_instance: # get a new instance so references do not get mixed up plugin_instance = plugin_instance.__class__.objects.get( pk=plugin_instance.pk ) plugin_instance.placeholder = target_placeholder # assign the pk of the generic plugin plugin_instance.pk = new_plugin.pk plugin_instance.id = new_plugin.pk # this is the OneToOne field plugin_instance.cmsplugin_ptr = new_plugin plugin_instance.language = target_language # tree information saved before plugin_instance.parent = new_plugin.parent plugin_instance.depth = new_plugin.depth plugin_instance.path = new_plugin.path -
EXPLOITING THE MODULAR NATURE OF DJANGO URLCONF Enter Apphooks The
way to integrate Django apps in the CMS tree -
APPHOOKS Tiny wrapper around urls.py class MyApp(CMSApp): name = _('My
app') urls = ['myapp.urls'] -
APPHOOKS 1. Add an apphook to the page 2. The
Apphook urls is injected in the global urlconf 3. magic happens -
APPHOOKS heavylift in cms/appresolver.py#L215-L245 for page_id in hooked_applications.keys(): resolver =
None for lang in hooked_applications[page_id].keys(): (app_ns, inst_ns), current_patterns, app = hooked_applications[pag if not resolver: resolver = AppRegexURLResolver(r'', 'app_resolver', app_name=a resolver.page_id = page_id if app.permissions: _set_permissions(current_patterns, app.exclude_permissions) resolver.url_patterns_dict[lang] = current_patterns app_patterns.append(resolver) APP_RESOLVERS.append(resolver) return app_patterns -
APPHOOKS 1. get the apphooks for lang in hooked_applications[page_id].keys(): (app_ns,
inst_ns), current_patterns, app =\ hooked_applications[page_id][lang] -
APPHOOKS 2. creates a custom resolver w/ apphook urls 3.
attach resolver to the page url (app_ns, inst_ns), current_patterns, app = hooked_applications[page_id][la if not resolver: resolver = AppRegexURLResolver( r'', 'app_resolver', app_name=app_ns, namespace=inst_ns ) resolver.page_id = page_id if app.permissions: _set_permissions(current_patterns, app.exclude_permissions) resolver.url_patterns_dict[lang] = current_patterns -
APPHOOKS 4. the apphooks resolvers are loaded into urlconf if
apphook_pool.get_apphooks(): urlpatterns = get_app_patterns() else: urlpatterns = [] urlpatterns.extend([ url(regexp, details, name='pages-details-by-slug'), url(r'^$', details, {'slug': ''}, name='pages-root'), ] -
IMPROVEMENT IN 3.1 NAMESPACES CONFIGURATION (via aldryn-apphooks-config) Detects current namespace
Loads a model related to the namespace Attach the model to the current view -
NAMESPACE CONFIGURATION def get_app_instance(request): ... with override(get_language_from_request(request, check_path=True namespace =
resolve(request.path_info).namespace config = app.get_config(namespace) ... def dispatch(self, request, *args, **kwargs): self.namespace, self.config = get_app_instance(request) -
NAMESPACE CONFIGURATION Example usage class PostListView(AppConfigMixin, ListView): model = Post
def get_template_names(self): template_path = self.config.template_prefix \ or 'djangocms_blog' return os.path.join(template_path, self.base_template_name) def get_paginate_by(self, queryset): return self.config.paginate_by or\ get_setting('PAGINATION') -
IMPROVEMENT IN 3.2 URLCONF RELOAD Into core Uses a middleware
1 DB hit per request Clears the urlconf modules and reload apphook patterns -
URLCONF RELOAD if 'cms.urls' in sys.modules: reload(sys.modules['cms.urls']) if urlconf is
None: urlconf = settings.ROOT_URLCONF if urlconf in sys.modules: reload(sys.modules[urlconf]) clear_app_resolvers() clear_url_caches() get_app_patterns() if new_revision is not None: set_local_revision(new_revision) -
BONUS INSTANCE AUTO SETUP (via djangocms-apphook-setup) Auto creation of a
page and apphook during projects startup 1 hit per project start -
AUTO SETUP Sample Apphook with namespace configuration auto setup configuration
class BlogApp(AutoCMSAppMixin, CMSConfigApp): name = _('Blog') urls = ['djangocms_blog.urls'] app_name = 'djangocms_blog' app_config = BlogConfig auto_setup = { ... 'page title': get_setting('AUTO_BLOG_TITLE'), 'namespace': get_setting('AUTO_NAMESPACE'), 'config_fields': {...}, 'config_translated_fields': {...}, } apphook_pool.register(BlogApp) BlogApp.setup() -
EXERCISE LEFT FOR THE READER Trace the call stack for
plugin rendering :)