Django as your data management framework Roman Imankulov Python Porto #11 in Blip | 30 Jan 2019 

Motivation 

What is Django 

What can Django give to me? • Project structure • Object-relational mapping (ORM) • Admin panel • Model-view-controller framework (MVC) 

Example github.com/imankulov/django-data-management-example 

What can Django give to me? • Project structure • Object-relational mapping (ORM) • Admin panel • Model-view-controller framework (MVC) 

project manage.py settings.py app app app urls.py 

project manage.py settings.py app app app urls.py admin.py models.py views.py templates … 

$ pip install Django $ django-admin startproject explore 

No content

What can Django give to me? • Project structure • Object-relational mapping (ORM) • Admin panel • Model-view-controller framework (MVC) 

CREATE TABLE groups ( id BIGINT PRIMARY KEY AUTOINCREMENT, name TEXT, urlname TEXT, description TEXT, created DATETIME, … ); > communities/models.py class Group(models.Model): name = models.CharField(…) urlname = models.CharField(…) description = models.TextField() created = models.DateTimeField() … class Meta: db_table = ‘groups’ 

No content

$ ./manage.py makemigrations 

$ ./manage.py makemigrations $ ./manage.py migrate 

$ ./manage.py makemigrations $ ./manage.py migrate $ ./manage.py inspectdb 

No content

Get a single object sql> SELECT * FROM groups WHERE id = 42 

Get a single object sql> SELECT * FROM groups WHERE id = 42 >>> Group.objects.get(id=42) 

Get a queryset sql> SELECT * FROM groups ... WHERE members > 400 ... AND created >= ‘2018-01-01’ ... ORDER BY members DESC ... LIMIT 10; 

Get a queryset sql> SELECT * FROM groups ... WHERE members > 400 ... AND created >= ‘2018-01-01’ ... ORDER BY members DESC ... LIMIT 10; >>> Group.objects.filter( members__gt=400, created__gt=datetime(2018, 1, 1) ).order_by(‘-members')[:10]
 , …]> 

Convert to DataFrame queryset = Group.objects.filter(…) df = pd.DataFrame.from_records(groups.values()) 

Convert to DataFrame queryset = Group.objects.filter(…) df = pd.DataFrame.from_records(groups.values()) city country ... urlname who 0 Lisbon PT ... ironhack-lisbon Ironhackers 1 Lisbon PT ... AI-Lisbon Members 2 Lisbon PT ... Lisbon-Artificial-Intelligence-Meetup Members 3 Lisbon PT ... Lisbon-Virtual-Currency-Meetup Members 4 Lisbon PT ... Zalando-Tech-Events-Lisbon Members 5 Lisbon PT ... Lisbon-Kaggle Kagglers 6 Lisbon PT ... Sketch-Design-Lisbon Members [7 rows x 13 columns] 

No content

What can Django give to me? • Project structure • Object-relational mapping (ORM) • Admin panel • Model-view-controller framework (MVC) 

No content

> communities/admin.py @admin.register(Group) class GroupAdmin(admin.ModelAdmin): list_display = ['name', ‘city’, ...] list_filter = ['country', 'city'] search_fields = ['name', 'organizer_name'] 

No content

No content

No content

What can Django give to me? • Project structure • Object-relational mapping (ORM) • Admin panel • Model-view-controller framework (MVC) 

What can Django give to me? • Project structure • Object-relational mapping (ORM) • Admin panel • Model-view-controller framework (MVC) Model-template-view framework (MTV) 

No content

No content

> urls.py urlpatterns = [ path('simple/', views.simple_dashboard), path('input/', views.input_dashboard), … ] GET /input/?limit=20&order_by=… > views.py def simple_dashboard(request): … 

> views.py def simple_dashboard(request): … return HttpResponse(…) 

> views.py @staff_member_required def simple_dashboard(request): … return HttpResponse(…) 

> views.py @staff_member_required def simple_dashboard(request): … return HttpResponse(…) 

> views.py def input_dashboard(request): form = InputForm(…) … df = pd.DataFrame.from_values(…) return render( request, ‘dashboard.html’, {…}) 

> views.py def input_dashboard(request): form = InputForm(…) … df = pd.DataFrame.from_values(…) return render( request, ‘dashboard.html’, {…}) > templates/dashboard.html {{ form.as_table }} {{ df.to_html|safe }} 

No content

What can Django give to me? • Project structure • Object-relational mapping (ORM) • Admin panel • Model-view-controller framework (MVC) 

Django hints for data people 

Django hints for data people • Can I use Django from Jupyter Notebook? • You can, but you need to initialize your environment properly to make it explicit which project we are working with right now 

Django hints for data people • Can I use Django from Jupyter Notebook? • You can, but you need to initialize your environment properly to make it explicit which project we are working with right now 

Django hints for data people • What is the easiest way to display a plot in my Django view? • Use Pygal (https://pygal.org) which draws plots to SVG and put them right in your templates 

Django hints for data people • What is the easiest way to display a plot in my Django view? • Use Pygal (https://pygal.org) which draws plots to SVG and put them right in your templates > views.py @staff_member_required def plot_dashboard(request): groups = Group.objects.order_by('-members')[:10] df = pd.DataFrame.from_records(groups.values()) bar = pygal.Bar(…) bar.x_labels = df.name bar.add('Members', df.members) return render(request, 'plot_dashboard.html', { 'df': df, 'bar': bar.render(is_unicode=True), }) 

Django hints for data people • What is the easiest way to display a plot in my Django view? • Use Pygal (https://pygal.org) which draws plots to SVG and put them right in your templates > views.py @staff_member_required def plot_dashboard(request): groups = Group.objects.order_by('-members')[:10] df = pd.DataFrame.from_records(groups.values()) bar = pygal.Bar(…) bar.x_labels = df.name bar.add('Members', df.members) return render(request, 'plot_dashboard.html', { 'df': df, 'bar': bar.render(is_unicode=True), }) > templates/plot_dashboard.html {{ bar|safe }} {{ df.to_html|safe }} 

Django hints for data people • What is the easiest way to display a plot in my Django view? • Use Pygal (https://pygal.org) which draws plots to SVG and put them right in your templates > views.py @staff_member_required def plot_dashboard(request): groups = Group.objects.order_by('-members')[:10] df = pd.DataFrame.from_records(groups.values()) bar = pygal.Bar(…) bar.x_labels = df.name bar.add('Members', df.members) return render(request, 'plot_dashboard.html', { 'df': df, 'bar': bar.render(is_unicode=True), }) > templates/plot_dashboard.html {{ bar|safe }} {{ df.to_html|safe }} 

Django hints for data people • Can I serve my Jupyter Notebooks with Django • It’s not straightforward, but you can convert them to HTML with nbconvert and serve this way 

Django hints for data people • Can I serve my Jupyter Notebooks with Django • It’s not straightforward, but you can convert them to HTML with nbconvert and serve this way > views.py class StringWriter(WriterBase): content = None def write(self, output, resources, **kw): self.content = output @staff_member_required def notebook(request): exporter = HTMLExporter( exclude_input=True, exclude_output_prompt=True) writer = StringWriter() app = NbConvertApp(writer=writer, exporter=exporter) app.convert_single_notebook(‘filename.ipynb’) return HttpResponse(writer.content) 

Django hints for data people • Can I serve my Jupyter Notebooks with Django • It’s not straightforward, but you can convert them to HTML with nbconvert and serve this way > views.py class StringWriter(WriterBase): content = None def write(self, output, resources, **kw): self.content = output @staff_member_required def notebook(request): exporter = HTMLExporter( exclude_input=True, exclude_output_prompt=True) writer = StringWriter() app = NbConvertApp(writer=writer, exporter=exporter) app.convert_single_notebook(‘filename.ipynb’) return HttpResponse(writer.content) 

Django hints for data people • How can I initiate long-running tasks from Django views? • Offload these tasks to workers. For example, use Celery 
 (http://celeryproject.org/) 

Questions?