Tweet
Tweet

Slide 1

Slide 1 text

DjangoCon US 2019 | William Vincent Search From the Ground Up William Vincent DjangoCon US 2019

Slide 2

Slide 2 text

DjangoCon US 2019 | William Vincent Just add search...

Slide 3

Slide 3 text

DjangoCon US 2019 | William Vincent /wsvincent/djangocon2019-search

Slide 4

Slide 4 text

DjangoCon US 2019 | William Vincent Who am I?

Slide 5

Slide 5 text

DjangoCon US 2019 | William Vincent Search Options Basic Filters Q Objects Full Text Search Hosted Service

Slide 6

Slide 6 text

DjangoCon US 2019 | William Vincent Search Magic City Search Website Massachusetts City Search Website City Search Website ● Boston, MA ● Worcester, MA ● Springfield, MA Form Results + Magic =

Slide 7

Slide 7 text

DjangoCon US 2019 | William Vincent Start a New Project

Slide 8

Slide 8 text

DjangoCon US 2019 | William Vincent Installation / Set Up $ pipenv install django==2.2.5 $ pipenv shell (env) $ django-admin startproject citysearch_project . (end) $ python manage.py migrate (end) $ python manage.py startapp cities

Slide 9

Slide 9 text

DjangoCon US 2019 | William Vincent INSTALLED_APPS # citysearch_project/settings.py INSTALLED_APPS = [ ... 'cities.apps.CitiesConfig', # new ]

Slide 10

Slide 10 text

DjangoCon US 2019 | William Vincent Models # cities/models.py from django.db import models class City(models.Model): name = models.CharField(max_length=100) state = models.CharField(max_length=100) class Meta: verbose_name_plural = 'cities'

Slide 11

Slide 11 text

DjangoCon US 2019 | William Vincent Migrations (env) $ python manage.py makemigrations cities (env) $ python manage.py migrate (env) $ python manage.py createsuperuser

Slide 12

Slide 12 text

DjangoCon US 2019 | William Vincent Admin # cities/admin.py from django.contrib import admin from .models import City class CityAdmin(admin.ModelAdmin): list_display = ('name', 'state') admin.site.register(City, CityAdmin)

Slide 13

Slide 13 text

DjangoCon US 2019 | William Vincent

Slide 14

Slide 14 text

DjangoCon US 2019 | William Vincent Django’s 4-Way Dance urls.py templates.html views.py models.py WEBPAGE!!!

Slide 15

Slide 15 text

DjangoCon US 2019 | William Vincent URLs (part 1) # citysearch_project/urls.py from django.contrib import admin from django.urls import path, include # new urlpatterns = [ path('admin/', admin.site.urls), path('', include('cities.urls')), # new ]

Slide 16

Slide 16 text

DjangoCon US 2019 | William Vincent URLs (part 2) # cities/urls.py from django.urls import path from .views import SearchResultsView, HomepageView urlpatterns = [ path('search/', SearchResultsView.as_view(), name='search_results'), path('', HomepageView.as_view(), name='home'), ]

Slide 17

Slide 17 text

DjangoCon US 2019 | William Vincent Views # cities/views.py from django.views.generic import TemplateView, ListView from .models import City class HomepageView(TemplateView): template_name = 'home.html' class SearchResultsView(ListView): model = City template_name = 'search_results.html'

Slide 18

Slide 18 text

DjangoCon US 2019 | William Vincent Template Structure Default (in app) citysearch_project |_ cities |_ templates |_ cities |_ home.html Custom (project-level) citysearch_project cities templates |_ home.html

Slide 19

Slide 19 text

DjangoCon US 2019 | William Vincent Template DIRS # citysearch_projects/settings.py TEMPLATES = [ { ... 'DIRS': [os.path.join(BASE_DIR, 'templates')], # new } ]

Slide 20

Slide 20 text

DjangoCon US 2019 | William Vincent Homepage Template # templates/home.html

HomePage

Slide 21

Slide 21 text

DjangoCon US 2019 | William Vincent Search Template # templates/search_results.html

Search Results

    {% for city in object_list %}
  • {{ city.name }}, {{ city.state }}
    • {% endfor %}

Slide 22

Slide 22 text

DjangoCon US 2019 | William Vincent Basic Search

Slide 23

Slide 23 text

DjangoCon US 2019 | William Vincent Filtering # cities/views.py class SearchResultsView(ListView): model = City template_name = 'search_results.html' def get_queryset(self): # new return City.objects.filter(name__icontains='Boston')

Slide 24

Slide 24 text

DjangoCon US 2019 | William Vincent Chaining Filters (ANDs) # cities/views.py class SearchResultsView(ListView): model = City template_name = 'search_results.html' def get_queryset(self): # new return City.objects.filter( name__icontains='Boston' ).exclude( state__icontains='NY' )

Slide 25

Slide 25 text

DjangoCon US 2019 | William Vincent Q Objects (ORs) # cities/views.py from django.db.models import Q # new ... class SearchResultsView(ListView): model = City template_name = 'search_results.html' def get_queryset(self): # new return City.objects.filter( Q(name__icontains='Boston') | Q(state__icontains='NY') )

Slide 26

Slide 26 text

DjangoCon US 2019 | William Vincent Forms san diego GET POST

Slide 27

Slide 27 text

DjangoCon US 2019 | William Vincent HTML Form # templates/home.html

HomePage

Slide 28

Slide 28 text

DjangoCon US 2019 | William Vincent Query not currently passed into View

Slide 29

Slide 29 text

DjangoCon US 2019 | William Vincent Add q to Queryset # cities/views.py class SearchResultsView(ListView): ... def get_queryset(self): # new query = self.request.GET.get('q') object_list = City.objects.filter( Q(name__icontains=query) | Q(state__icontains=query) ) return object_list

Slide 30

Slide 30 text

DjangoCon US 2019 | William Vincent Django Forms # cities/forms.py from django import forms class SearchForm(forms.Form): q = forms.CharField(label='Search', max_length=50)

Slide 31

Slide 31 text

DjangoCon US 2019 | William Vincent View with Django Form # cities/views.py from django.views.generic import FormView, ListView from .forms import SearchForm from .models import City class HomepageView(FormView): template_name = 'home.html' form_class = SearchForm ...

Slide 32

Slide 32 text

DjangoCon US 2019 | William Vincent Template with Django Form # templates/home.html

Homepage

{{ form }}

Slide 33

Slide 33 text

DjangoCon US 2019 | William Vincent Full-Text Search

Slide 34

Slide 34 text

DjangoCon US 2019 | William Vincent Search Options Basic Filters Q Objects Full Text Search Hosted Service

Slide 35

Slide 35 text

DjangoCon US 2019 | William Vincent Full-Text Search Search Documents that satisfy a Query and sort by Relevance. ● PostgreSQL support since 8.3 (2008) ● Django support since version 1.10 (2016) led by Marc Tamlyn

Slide 36

Slide 36 text

DjangoCon US 2019 | William Vincent Full Text Search Features ● Rankings ● Indexing ● Phrase search ● Stop words ● Stemming ● Accent / multiple language support ● JSON[B] support

Slide 37

Slide 37 text

DjangoCon US 2019 | William Vincent Pre-Processing Documents -> Tokens -> Lexemes

Slide 38

Slide 38 text

DjangoCon US 2019 | William Vincent PostgreSQL Data Types tsvector: preprocessed documents tsquery: processed queries

Slide 39

Slide 39 text

DjangoCon US 2019 | William Vincent tsvector Document: 'The quick brown fox jumps over the lazy dog.' tsvector ------------------------------------------------------------------------------------- 'brown':3 'dog':9 'fox':4 'jump':5 'lazy':8 'quick:2'

Slide 40

Slide 40 text

DjangoCon US 2019 | William Vincent tsquery List of words checked against normalized tsvector 'The quick brown fox jumps over the lazy dog.' Query Match? (using @@ operator) 'dog' true 'dogs' true 'dogfood' false 'jumping' true

Slide 41

Slide 41 text

DjangoCon US 2019 | William Vincent tsquery operators 'The quick brown fox jumps over the lazy dog.' Operator tsquery Result? @@ (match) 'dogs foxes' true & (and) 'dog & fox' true | (or) 'dog | cat' true ! (negation) '!pony ' true

Slide 42

Slide 42 text

DjangoCon US 2019 | William Vincent django.contrib.postgres.search SearchVector = search multiple fields SearchQuery = add stemming and/or stop words SearchRank = apply rank to result of documents Weighted Queries = add weights SearchVectorField = performance upgrade with manual trigger

Slide 43

Slide 43 text

DjangoCon US 2019 | William Vincent SearchVector # cities/views.py from django.contrib.postgres.search import SearchVector ... class SearchResultsView(ListView): ... def get_queryset(self): query = self.request.GET.get('q') object_list = City.objects.annotate( search=SearchVector('name', 'state'), ).filter(search=query) return object_list

Slide 44

Slide 44 text

DjangoCon US 2019 | William Vincent SearchQuery # cities/views.py from django.contrib.postgres.search import SearchVector, SearchQuery def get_queryset(self): query = self.request.GET.get('q') object_list = City.objects.annotate( search=SearchVector('name', 'state'), ).filter(search=SearchQuery(query)) return object_list

Slide 45

Slide 45 text

DjangoCon US 2019 | William Vincent SearchRank # cities/views.py from django.contrib.postgres.search import SearchVector, SearchQuery, SearchRank def get_queryset(self): query = self.request.GET.get('q') vector = SearchVector('state') search_query = SearchQuery(query) object_list = City.objects.annotate( rank=SearchRank(vector, search_query) ).order_by('-rank') return object_list

Slide 46

Slide 46 text

DjangoCon US 2019 | William Vincent Resources Source Code https://github.com/wsvincent/djangocon2019-search Slides https://tinyurl.com/djangocon2019-search PostgreSQL Docs: Full Text Search https://www.postgresql.org/docs/current/textsearch.html