Developing the rpostgisLT package

An R package for managing animal movement data with PostGIS and R

Transcript

1. Developing the rpostgisLT package An R package for managing animal movement data with PostGIS and R Balázs Dukai 2018-03-20 1 / 20
2. Once upon a time... There was a GSoC 2016 project where my application was accepted. Then we realisied that we work well with each other, so why not continue in a GSoC 2017 project? The project idea came from the Movement and Habitat Laboratory (MabLab) at the University of Florida’s Fort Lauderdale Research and Education Center. 2 / 20
3. Once upon a time... There was a GSoC 2016 project where my application was accepted. Then we realisied that we work well with each other, so why not continue in a GSoC 2017 project? The project idea came from the Movement and Habitat Laboratory (MabLab) at the University of Florida’s Fort Lauderdale Research and Education Center. MabLab a quantitative ecology lab focusing on spatial ecology — and in particular movement 2 / 20
4. Once upon a time... There was a GSoC 2016 project where my application was accepted. Then we realisied that we work well with each other, so why not continue in a GSoC 2017 project? The project idea came from the Movement and Habitat Laboratory (MabLab) at the University of Florida’s Fort Lauderdale Research and Education Center. MabLab a quantitative ecology lab focusing on spatial ecology — and in particular movement needed good tools to work with the data that they collect on the field everyday (mainly with GPS trackers) 2 / 20
5. South Florida raccoon inspecting photographer for a potential meal. Credit Paxson Woelber 3 / 20
6. What is a trajectory? According to adehabitat1 [1] Calenge, C., Dray, S., & Royer-Carenzi, M. (2009). The concept of animals' trajectories from a data analysis perspective. Ecological informatics, 4(1), 34-41. 4 / 20
7. What is a trajectory? According to adehabitat1 locations -> steps -> bursts -> trajectory [1] Calenge, C., Dray, S., & Royer-Carenzi, M. (2009). The concept of animals' trajectories from a data analysis perspective. Ecological informatics, 4(1), 34-41. 4 / 20
8. What is a trajectory? According to adehabitat1 locations -> steps -> bursts -> trajectory With time stamps (e.g. GPS) -> type II, [1] Calenge, C., Dray, S., & Royer-Carenzi, M. (2009). The concept of animals' trajectories from a data analysis perspective. Ecological informatics, 4(1), 34-41. 4 / 20
9. What is a trajectory? According to adehabitat1 locations -> steps -> bursts -> trajectory With time stamps (e.g. GPS) -> type II, regular time lag (e.g. every 4 hours) irregular time lag [1] Calenge, C., Dray, S., & Royer-Carenzi, M. (2009). The concept of animals' trajectories from a data analysis perspective. Ecological informatics, 4(1), 34-41. 4 / 20
10. What is a trajectory? According to adehabitat1 locations -> steps -> bursts -> trajectory With time stamps (e.g. GPS) -> type II, regular time lag (e.g. every 4 hours) irregular time lag or without (e.g. tracks in snow) -> type I [1] Calenge, C., Dray, S., & Royer-Carenzi, M. (2009). The concept of animals' trajectories from a data analysis perspective. Ecological informatics, 4(1), 34-41. 4 / 20
11. What is a trajectory? According to adehabitat1 locations -> steps -> bursts -> trajectory With time stamps (e.g. GPS) -> type II, regular time lag (e.g. every 4 hours) irregular time lag or without (e.g. tracks in snow) -> type I Plus additional attributes at each location (infolocs) [1] Calenge, C., Dray, S., & Royer-Carenzi, M. (2009). The concept of animals' trajectories from a data analysis perspective. Ecological informatics, 4(1), 34-41. 4 / 20
12. What is a trajectory? According to adehabitat1 locations -> steps -> bursts -> trajectory With time stamps (e.g. GPS) -> type II, regular time lag (e.g. every 4 hours) irregular time lag or without (e.g. tracks in snow) -> type I Plus additional attributes at each location (infolocs) (Re)locations can be missing e.g. no GPS signal at the moment of relocation [1] Calenge, C., Dray, S., & Royer-Carenzi, M. (2009). The concept of animals' trajectories from a data analysis perspective. Ecological informatics, 4(1), 34-41. 4 / 20
13. Locations and Steps 5 / 20
14. Parameters of a step 6 / 20
15. Bursts (of movement modes) 7 / 20
16. A trajectory Can be split and combined from several "parts" to study the different activities of an animal. 8 / 20
17. The ltraj1 class models the trajectories And an ltraj object is just bunch of data frames, plus a few attributes [1] adehabitatLT package 9 / 20
18. An ltraj goes into Postgres -> pgtraj Conceptually, a pgtraj and an ltraj are equivalent. all.equal(pgtraj, ltraj) == TRUE R POSTGRESQL PGTRAJ LTRAJ PGTRAJ LTRAJ PGTRAJ LTRAJ TRAJECTORIES SCHEMA 10 / 20
19. A minimal trajectory schema 11 / 20
20. The rules of a pgtraj 12 / 20
21. The rules of a pgtraj each pgtraj has a unique name 12 / 20
22. The rules of a pgtraj each pgtraj has a unique name a pgtraj has at least one named burst and related to at least one named animal 12 / 20
23. The rules of a pgtraj each pgtraj has a unique name a pgtraj has at least one named burst and related to at least one named animal within a pgtraj the burst names must be unique, but they can repeat across pgtrajes 12 / 20
24. The rules of a pgtraj each pgtraj has a unique name a pgtraj has at least one named burst and related to at least one named animal within a pgtraj the burst names must be unique, but they can repeat across pgtrajes there is no restriction on the duplication of animal names, therefore it is possible to store modified pgtrajes of the same animal in the same traj schema 12 / 20
25. The rules of a pgtraj each pgtraj has a unique name a pgtraj has at least one named burst and related to at least one named animal within a pgtraj the burst names must be unique, but they can repeat across pgtrajes there is no restriction on the duplication of animal names, therefore it is possible to store modified pgtrajes of the same animal in the same traj schema it is also possible that a single step is related to multiple bursts or pgtrajes 12 / 20
26. The rules of a pgtraj each pgtraj has a unique name a pgtraj has at least one named burst and related to at least one named animal within a pgtraj the burst names must be unique, but they can repeat across pgtrajes there is no restriction on the duplication of animal names, therefore it is possible to store modified pgtrajes of the same animal in the same traj schema it is also possible that a single step is related to multiple bursts or pgtrajes an infolocs table relate a set of data to each step of a single pgtraj and the table follows the naming convention infolocs_(pgtraj_name). 12 / 20
27. The core functions 13 / 20
28. And explorePgtraj() Shiny + Leaflet + PostGIS + sf package 14 / 20
29. And explorePgtraj() Shiny + Leaflet + PostGIS + sf package A close call for a QGIS plugin, but wanted to stay in R Keyboard navigation and responsiveness were important Geometry is dynamically retrieved from Postgres Materialised views and query optimisation for visualisation 14 / 20
30. And explorePgtraj() Shiny + Leaflet + PostGIS + sf package A close call for a QGIS plugin, but wanted to stay in R Keyboard navigation and responsiveness were important Geometry is dynamically retrieved from Postgres Materialised views and query optimisation for visualisation 14 / 20
31. Lessons learned 15 / 20
32. Lessons learned Leaflet is powerful but is not the best for dynamic trajectory visualisation 15 / 20
33. Lessons learned Leaflet is powerful but is not the best for dynamic trajectory visualisation Need performance from a database for visualisation? Consider materialising your data 15 / 20
34. Lessons learned Leaflet is powerful but is not the best for dynamic trajectory visualisation Need performance from a database for visualisation? Consider materialising your data A thorough requirements analysis (e.g. MoSCoW) is a good way to create a common understanding of the project and avoid the "let's just quickly implement this cool extra feature..." 15 / 20
35. Lessons learned Leaflet is powerful but is not the best for dynamic trajectory visualisation Need performance from a database for visualisation? Consider materialising your data A thorough requirements analysis (e.g. MoSCoW) is a good way to create a common understanding of the project and avoid the "let's just quickly implement this cool extra feature..." README-driven development -> write the README first, then the code 15 / 20
36. Lessons learned Leaflet is powerful but is not the best for dynamic trajectory visualisation Need performance from a database for visualisation? Consider materialising your data A thorough requirements analysis (e.g. MoSCoW) is a good way to create a common understanding of the project and avoid the "let's just quickly implement this cool extra feature..." README-driven development -> write the README first, then the code Trying to make a GIS with R? Maybe rather bring R into a GIS (e.g. QGIS). There are benefits to each approach. Who will use your tool? What do you expect from them to know? 15 / 20
37. Lessons learned Leaflet is powerful but is not the best for dynamic trajectory visualisation Need performance from a database for visualisation? Consider materialising your data A thorough requirements analysis (e.g. MoSCoW) is a good way to create a common understanding of the project and avoid the "let's just quickly implement this cool extra feature..." README-driven development -> write the README first, then the code Trying to make a GIS with R? Maybe rather bring R into a GIS (e.g. QGIS). There are benefits to each approach. Who will use your tool? What do you expect from them to know? Testing, testing, testing...But watch out for the overhead in maintenance 15 / 20
38. Lessons learned Leaflet is powerful but is not the best for dynamic trajectory visualisation Need performance from a database for visualisation? Consider materialising your data A thorough requirements analysis (e.g. MoSCoW) is a good way to create a common understanding of the project and avoid the "let's just quickly implement this cool extra feature..." README-driven development -> write the README first, then the code Trying to make a GIS with R? Maybe rather bring R into a GIS (e.g. QGIS). There are benefits to each approach. Who will use your tool? What do you expect from them to know? Testing, testing, testing...But watch out for the overhead in maintenance Who will maintain the code? 15 / 20
39. <!-- Breeding pair of Wood Storks. Credit Simona Picardi 16 / 20
40. Demo 17 / 20
41. Connect to a PostgreSQL database Needs PostGIS, an existing database and user with CREATE privilege in the schema library(rpostgisLT) conn_demo <- RPostgreSQL::dbConnect(RPostgreSQL::PostgreSQL(), dbname = "rpglt_demo", user = "demo", host = "localhost") 18 / 20
42. Load your data GPS records are stored in the example_data.stork_gps table. gps_data_animals_id animal_id acquisition_time geom pgtraj burst 414354 475270 2004-05-29 13:00:00 POINT (-80.145 26.76717) 2006 summer_475270 428683 475280 2004-05-29 13:00:00 POINT (-80.145 26.76717) 2006 summer_475280 433580 475290 2004-05-29 13:00:00 POINT (-80.1455 26.768) 2006 summer_475290 435865 475300 2004-05-29 13:00:00 POINT (-80.145 26.7675) 2006 summer_475300 asPgtraj(conn = conn_demo, relocations_table = c("example_data", "stork_gps"), schema = "stork_traj", pgtrajs = "pgtraj", animals = "animal_id", bursts = "burst", relocations = "geom", timestamps = "acquisition_time", rids = "gps_data_animals_id") 19 / 20
43. Start the interactive tool explorePgtraj(conn = conn_demo, schema = "stork_traj", pgtraj = "2006") And disconnect afterwards RPostgreSQL::dbDisconnect(conn_demo) 20 / 20