Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Maps with Django (and PostGIS) - Citus Con 2023

Maps with Django (and PostGIS) - Citus Con 2023

-- A talk I gave at Citus Con 2023

Keeping in mind the Pythonic principle that “simple is better than complex” we’ll see how to create a web map with the Python-based web framework Django using its GeoDjango module, storing geographic data in your local database on which to run geospatial queries.

https://www.paulox.net/2023/04/18/citus-con-2023/

Paolo Melchiorre

March 21, 2023
Tweet

More Decks by Paolo Melchiorre

Other Decks in Technology

Transcript

  1. MAPS WITH DJANGO
    (AND POSTGIS)
    PAOLO MELCHIORRE ~ @pauloxnet

    View full-size slide

  2. Paolo Melchiorre ~ @pauloxnet
    2

    View full-size slide

  3. Paolo Melchiorre ~ @pauloxnet
    @pauloxnet
    • CTO @ 20tab
    • Software engineer
    • Python developer
    • Django contributor
    • Hiking enthusiast
    Paolo Melchiorre
    3
    © 2022 Bartek Pawlik (CC BY-NC-SA) - DjangoCon US

    View full-size slide

  4. Paolo Melchiorre ~ @pauloxnet
    1.883 m
    0 km 27 km
    0 km
    4
    © 2020 Paolo Melchiorre (CC BY-SA) Adriatic Sea from the Italian Apennines

    View full-size slide

  5. Paolo Melchiorre ~ @pauloxnet
    • Static or Dynamic
    • Interactive or view-only
    • Raster or Vector tiles
    • Spatial databases
    • Javascript library
    Web maps features
    5

    View full-size slide

  6. Paolo Melchiorre ~ @pauloxnet
    6
    Web mapping
    <… process of using the maps
    delivered by Geographic Information Systems (GIS)
    on the Internet …=

    View full-size slide

  7. Paolo Melchiorre ~ @pauloxnet
    django
    7
    © 1946 William Gottlieb (Public Domain) - Django Reinhardt at the Aquarium jazz club in New York

    View full-size slide

  8. Paolo Melchiorre ~ @pauloxnet
    8
    Requirements
    $ python3.11 -m venv ~/.mymap
    $ . ~/.mymap/bin/activate
    $ python -m pip install django~=4.1

    Successfully installed … django-4.1.7 …

    View full-size slide

  9. Paolo Melchiorre ~ @pauloxnet
    9
    Creating the 8mymap9 project
    $ cd ~/projects
    $ python -m django startproject mymap
    mymap/
    ├── manage.py
    └── mymap
    ├── asgi.py
    ├── __init__.py
    ├── settings.py
    ├── urls.py
    └── wsgi.py

    View full-size slide

  10. Paolo Melchiorre ~ @pauloxnet
    10
    Creating the 8markers9 app
    $ cd mymap
    $ python -m django startapp markers
    markers/
    ├── admin.py
    ├── apps.py
    ├── __init__.py
    ├── migrations
    │ └── __init__.py
    ├── models.py
    ├── tests.py
    └── views.py

    View full-size slide

  11. Paolo Melchiorre ~ @pauloxnet
    11
    Activating the 8markers9 app
    # mymap/mymap/settings.py
    INSTALLED_APPS = [
    "django.contrib.admin",
    "django.contrib.auth",
    "django.contrib.contenttypes",
    "django.contrib.sessions",
    "django.contrib.messages",
    "django.contrib.staticfiles",
    "markers",
    ]

    View full-size slide

  12. Paolo Melchiorre ~ @pauloxnet
    12
    Adding a template view
    # mymap/markers/views.py
    from django.views.generic import TemplateView
    class MarkersMapView(TemplateView):
    template_name = "map.html"

    View full-size slide

  13. Paolo Melchiorre ~ @pauloxnet
    13
    Adding the 8map9 template




    Markers Map
    content="width=device-width, initial-scale=1.0" />



    View full-size slide

  14. Paolo Melchiorre ~ @pauloxnet
    14
    Adding 8markers9 urls
    # mymap/markers/urls.py
    from django.urls import path
    from markers.views import MarkersMapView
    app_name = "markers"
    urlpatterns = [
    path("map/", MarkersMapView.as_view()),
    ]

    View full-size slide

  15. Paolo Melchiorre ~ @pauloxnet
    15
    Updating 8mymap9 urls
    # mymap/mymap/urls.py
    from django.contrib import admin
    from django.urls import include, path
    urlpatterns = [
    path("admin/", admin.site.urls),
    path("markers/", include("markers.urls")),
    ]

    View full-size slide

  16. Paolo Melchiorre ~ @pauloxnet
    2.068 m
    0 km 27 km
    2 km
    16
    © 2020 Paolo Melchiorre (CC BY-SA) Sunrise panorama on the Blockhaus

    View full-size slide

  17. Paolo Melchiorre ~ @pauloxnet
    • JavaScript library for maps
    • Free Software
    • Desktop & Mobile friendly
    • Light (~42 KB of gzipped JS)
    • Well documented
    Leaflet
    17

    View full-size slide

  18. Paolo Melchiorre ~ @pauloxnet
    18
    Updating the 8map9 template - header

    {% load static %}


    href="{% static 'map.css' %}" />
    href="https://unpkg.com/leaflet/dist/leaflet.css"/>
    crossorigin>

    View full-size slide

  19. Paolo Melchiorre ~ @pauloxnet
    19
    Updating the 8map9 template - body

    {% load static %}








    View full-size slide

  20. Paolo Melchiorre ~ @pauloxnet
    20
    Adding the 8map9 CSS
    /* mymap/markers/static/map.css */
    html,
    body {
    height: 100%;
    margin: 0;
    }
    #map {
    height: 100%;
    width: 100%;
    }

    View full-size slide

  21. Paolo Melchiorre ~ @pauloxnet
    21
    Adding the 8map9 JavaScript
    // mymap/markers/static/map.js
    const osm = "https://www.osm.org/copyright";
    const copy = `© OpenStreetMap`;
    const url = "https://{s}.tile.osm.org/{z}/{x}/{y}.png";
    const layer = L.tileLayer(url, { attribution: copy });
    const map = L.map("map", { layers: [layer] });
    map.fitWorld();

    View full-size slide

  22. Paolo Melchiorre ~ @pauloxnet
    22
    Show the empty web map
    $ python -m manage runserver
    $ python -m webbrowser -t localhost:8000/markers/map

    View full-size slide

  23. Paolo Melchiorre ~ @pauloxnet
    23

    View full-size slide

  24. Paolo Melchiorre ~ @pauloxnet
    2.117 m
    0 km 27 km
    6 km
    24
    © 2020 Paolo Melchiorre (CC BY-SA) Trail signs near the Sella Acquaviva fountain

    View full-size slide

  25. Paolo Melchiorre ~ @pauloxnet
    25
    GeoDjango
    django.contrib.gis (v1.0 ~2008)
    Fields, backends, queries, admin, …
    Spatialite backend (v1.1 ~2009)
    Multiple backends (v1.2 ~2010)
    OpenLayers-based widgets (v1.6 ~2013)
    GeoJSON serializer (v1.8 ~2015)
    GeoIP2 Geolocation (v1.9 ~2016)

    View full-size slide

  26. Paolo Melchiorre ~ @pauloxnet
    • OSGeo library
    • Free Software
    • Read/Write geospatial data
    • Raster/Vector formats
    • Command line interface
    GDAL
    26

    View full-size slide

  27. Paolo Melchiorre ~ @pauloxnet
    27
    Installing GDAL
    $ sudo apt install gdal-bin

    View full-size slide

  28. Paolo Melchiorre ~ @pauloxnet
    28
    Activating GeoDjango
    # mymap/mymap/settings.py
    INSTALLED_APPS = [
    "django.contrib.admin",
    "django.contrib.auth",
    "django.contrib.contenttypes",
    "django.contrib.sessions",
    "django.contrib.messages",
    "django.contrib.staticfiles",
    "django.contrib.gis",
    "markers",
    ]

    View full-size slide

  29. Paolo Melchiorre ~ @pauloxnet
    • SQLite spatial extension
    • Vector geodatabase functions
    • Free Software
    • Simple architecture
    • Single file
    SpatiaLite
    29

    View full-size slide

  30. Paolo Melchiorre ~ @pauloxnet
    30
    Installing SpatiaLite
    $ sudo apt install libsqlite3-mod-spatialite

    View full-size slide

  31. Paolo Melchiorre ~ @pauloxnet
    31
    Activating SpatiaLite
    # mymap/settings.py
    DATABASES = {"default": {
    "ENGINE": "django.contrib.gis.db.backends.spatialite",
    "NAME": BASE_DIR / "db.sqlite3",
    }}

    View full-size slide

  32. Paolo Melchiorre ~ @pauloxnet
    32
    Adding the Marker model
    # mymap/markers/models.py
    from django.contrib.gis.db import models
    class Marker(models.Model):
    name = models.CharField(max_length=255)
    location = models.PointField()
    def __str__(self):
    return self.name

    View full-size slide

  33. Paolo Melchiorre ~ @pauloxnet
    33
    Updating the database
    $ python -m manage makemigrations
    $ python -m manage migrate

    View full-size slide

  34. Paolo Melchiorre ~ @pauloxnet
    34
    Adding the Marker admin
    # mymap/markers/admin.py
    from django.contrib.gis import admin
    from markers.models import Marker
    @admin.register(Marker)
    class MarkerAdmin(admin.GISModelAdmin):
    list_display = ("name", "location")

    View full-size slide

  35. Paolo Melchiorre ~ @pauloxnet
    35
    Adding some markers
    $ python -m manage createsuperuser
    $ python -m manage runserver
    $ python -m webbrowser -t http://localhost:8000/admin

    View full-size slide

  36. Paolo Melchiorre ~ @pauloxnet
    36

    View full-size slide

  37. Paolo Melchiorre ~ @pauloxnet
    37
    Updating the view - part 1
    # markers/views.py
    import json
    from django.core.serializers import serialize
    from django.views.generic.base import TemplateView
    from markers.models import Marker
    class MarkersMapView(TemplateView):
    template_name = "map.html"
    def get_context_data(self, **kwargs):
    ...

    View full-size slide

  38. Paolo Melchiorre ~ @pauloxnet
    38
    Updating the view - part 2
    # markers/views.py
    class MarkersMapView(TemplateView):
    ...
    def get_context_data(self, **kwargs):
    context = super().get_context_data(**kwargs)
    markers = Marker.objects.all()
    geojson = serialize("geojson", markers)
    context["markers"] = json.loads(geojson)
    return context

    View full-size slide

  39. Paolo Melchiorre ~ @pauloxnet
    39
    Generated GeoJSON
    {"type": "FeatureCollection", "crs": {
    "type": "name", "properties": {"name": "EPSG:4326"}},
    "features": [{
    "type": "Feature",
    "properties": { "name": "Monte Amaro", "pk": "1" },
    "geometry": {
    "type": "Point", "coordinates": [14.085, 42.086]
    }
    }]
    }

    View full-size slide

  40. Paolo Melchiorre ~ @pauloxnet
    40
    Inserting markers in the template

    {% load static %}




    {{ markers|json_script:"markers-data" }}




    View full-size slide

  41. Paolo Melchiorre ~ @pauloxnet
    41
    Rendering all markers in the map
    // markers/static/map.js
    // ...
    // map.fitWorld();
    const data = document.getElementById("markers-data");
    let feature = L.geoJSON(JSON.parse(data.textContent))
    .bindPopup(function (layer) {
    return layer.feature.properties.name;
    })
    .addTo(map);
    map.fitBounds(feature.getBounds());

    View full-size slide

  42. Paolo Melchiorre ~ @pauloxnet
    42
    Show the populated web map
    $ python -m manage runserver
    $ python -m webbrowser -t localhost:8000/markers/map

    View full-size slide

  43. Paolo Melchiorre ~ @pauloxnet
    43

    View full-size slide

  44. Paolo Melchiorre ~ @pauloxnet
    2.450 m
    0 km 27 km
    7 km
    44
    © 2020 Paolo Melchiorre (CC BY-SA) Trail signs near the Fusco bivouac

    View full-size slide

  45. Paolo Melchiorre ~ @pauloxnet
    • PostgreSQL extension
    • Best* GeoDjango backend
    • Spatial data types
    • Spatial indexing
    • Spatial functions
    PostGIS
    45

    View full-size slide

  46. Paolo Melchiorre ~ @pauloxnet
    46
    GeoDjango Compatibility Table
    ● Spatial Lookups (18/31)
    ● Database functions (15/32)
    ● Aggregate Functions (0/5)
    MariaDB
    49%
    ● Spatial Lookups (31/31)
    ● Database functions (32/32)
    ● Aggregate Functions (5/5)
    PostGIS
    100%
    ● Spatial Lookups (22/31)
    ● Database functions (30/32)
    ● Aggregate Functions (4/5)
    Spatialite
    82%
    ● Spatial Lookups (18/31)
    ● Database functions (16/32)
    ● Aggregate Functions (0/5)
    MySQL
    50%

    View full-size slide

  47. Paolo Melchiorre ~ @pauloxnet
    47
    Installing PostgreSQL C client library
    $ sudo apt install libpq5

    View full-size slide

  48. Paolo Melchiorre ~ @pauloxnet
    48
    Requirements
    # requirements.txt
    django-filter~=22.1.0
    djangorestframework-gis~=1.0.0
    djangorestframework~=3.14.0
    django~=4.1.0
    psycopg2-binary~=2.9.0

    View full-size slide

  49. Paolo Melchiorre ~ @pauloxnet
    49
    Installing requirements
    $ python -m pip install -r requirements.txt

    View full-size slide

  50. Paolo Melchiorre ~ @pauloxnet
    50
    Activating PostGIS
    # mymap/mymap/settings.py
    DATABASES = {"default": {
    "ENGINE": "django.contrib.gis.db.backends.postgis",
    "HOST": "database",
    "NAME": "mymap",
    "PASSWORD": "password",
    "PORT": 5432,
    "USER": "postgres",
    }}

    View full-size slide

  51. Paolo Melchiorre ~ @pauloxnet
    Applying migrations to PostgreSQL
    51
    $ python -m manage migrate

    View full-size slide

  52. Paolo Melchiorre ~ @pauloxnet
    52
    Activating Django REST Framework
    # mymap/settings.py
    INSTALLED_APPS = [
    ...
    "rest_framework",
    "rest_framework_gis",
    "markers",
    ]

    View full-size slide

  53. Paolo Melchiorre ~ @pauloxnet
    53
    Adding the Marker serializer
    # mymap/markers/serializers.py
    from rest_framework_gis import serializers as srlzs
    from markers.models import Marker
    class MarkerSerializer(srlzs.GeoFeatureModelSerializer):
    class Meta:
    fields = ("id", "name")
    geo_field = "location"
    model = Marker

    View full-size slide

  54. Paolo Melchiorre ~ @pauloxnet
    54
    Adding the Marker viewset
    # mymap/markers/viewsets.py
    from rest_framework import viewsets
    from rest_framework_gis import filters
    from markers.models import Marker
    from markers.serializers import MarkerSerializer
    class MarkerViewSet(viewsets.ReadOnlyModelViewSet):
    bbox_filter_field = "location"
    filter_backends = (filters.InBBoxFilter,)
    queryset = Marker.objects.all()
    serializer_class = MarkerSerializer

    View full-size slide

  55. Paolo Melchiorre ~ @pauloxnet
    55
    Adding API 8markers9 urls
    # mymap/markers/api.py
    from rest_framework import routers
    from markers.viewsets import MarkerViewSet
    router = routers.DefaultRouter()
    router.register(r"markers", MarkerViewSet)
    urlpatterns = router.urls

    View full-size slide

  56. Paolo Melchiorre ~ @pauloxnet
    56
    Updating 8mymap9 urls
    # mymap/mymap/urls.py
    from django.contrib import admin
    from django.urls import include, path
    urlpatterns = [
    path("admin/", admin.site.urls),
    path("api/", include("markers.api")),
    path("markers/", include("markers.urls")),
    ]

    View full-size slide

  57. Paolo Melchiorre ~ @pauloxnet
    57
    Trying to locate the user
    // mymap/markers/static/map.js
    // ...
    map.locate()
    .on("locationfound", (e) => map.setView(e.latlng, 8))
    .on("locationerror", () => map.setView([0, 0], 5));
    // ...

    View full-size slide

  58. Paolo Melchiorre ~ @pauloxnet
    58
    Fetch required markers
    // mymap/markers/static/map.js
    // ...
    async function load_markers() {
    const bounds = map.getBounds().toBBoxString();
    const markers_url = `/api/markers/?in_bbox=${bounds}`;
    const response = await fetch(markers_url);
    const geojson = await response.json();
    return geojson;
    }
    // ...

    View full-size slide

  59. Paolo Melchiorre ~ @pauloxnet
    59
    Rendering markers incrementally
    // mymap/markers/static/map.js
    // ...
    async function render_markers() {
    const markers = await load_markers();
    L.geoJSON(markers)
    .bindPopup((layer) => layer.feature.properties.name)
    .addTo(map);
    }
    map.on("moveend", render_markers);

    View full-size slide

  60. Paolo Melchiorre ~ @pauloxnet
    60

    View full-size slide

  61. Paolo Melchiorre ~ @pauloxnet
    61
    Demo

    View full-size slide

  62. Paolo Melchiorre ~ @pauloxnet
    2.793 m
    0 km 27 km
    14 km
    62
    © 2020 Paolo Melchiorre (CC BY-SA) Summit of Monte Amaro (2.793 meters)

    View full-size slide

  63. Paolo Melchiorre ~ @pauloxnet
    63
    What9s next
    • Markers customization
    • Relational filtering
    • Clustering frontend/backend
    • Geocoding services
    • …

    View full-size slide

  64. Paolo Melchiorre ~ @pauloxnet
    64
    Tips
    • docs in djangoproject.com
    • details in postgis.net
    • source code in github.com
    • questions in gis.stackexchange.com

    View full-size slide

  65. Paolo Melchiorre ~ @pauloxnet
    65
    License
    CC BY-SA 4.0
    This work is licensed under
    a Creative Commons
    Attribution-ShareAlike 4.0
    International License.

    View full-size slide

  66. Paolo Melchiorre ~ @pauloxnet
    @20tab
    20tab
    20tab
    [email protected]
    20tab.com
    66

    View full-size slide

  67. Paolo Melchiorre ~ @pauloxnet
    @pauloxnet
    paolomelchiorre
    pauloxnet
    www.paulox.net
    67
    @[email protected]

    View full-size slide