Occurrence data at the INBO: opening up our data publication workflow

Talk at the TDWG 2014 annual conference in Jönköping, Sweden - October 30, 2014.

Our workflow for publishing biodiversity data, issues we encountered and some ways forward.

Transcript

1. Occurrence  data  at  the  INBO   Opening  up  our  data  publica5on  workflow   Peter  Desmet  &  Dimitri  Brosens  
3. Biodiversity  research   Suppor5ng  policy,  nature  management  &   interna5onal  repor5ng  
4. Monitoring   Resul5ng  in  lots  of  occurrence  data  
5. Human  observa5ons   Going  back  20  years  and  more  
6. Machine  observa5ons   Mainly  via  lifewatch.inbo.be  
7. Publishing  data   Since  2011,  through  data.inbo.be/ipt  
8. 15  datasets   4.6  million  observa5ons  
10. Open  data   CC0  +  Norms  for  data  use  
11. Our  publica5on  workflow   Is  not  rocket  science  
12. Create  GitHub  repository   github.com/LifeWatchINBO  
13. Create  test  resource   On  our  sandbox  IPT  
14. Create  Darwin  Core  view   All  our  data  are  (imported)  on  SQL  Server  
15. Write  metadata   As  a  data  paper,  in  Markdown  
16. Document  issues   For  data  and  metadata  
17. Resolve  data  issues   Google  Refine  >  Database  or  mapping  
18. Review  metadata   With  final  approval  by  the  researchers  
19. Migrate  to  produc5on   Copy/pas5ng  
20. Publish  &  register   And  relax…   Tripel  Karmeliet  Beer  ©  Johan  Martens,  licensed  under  CC  BY-­‐NC-­‐SA  2.0  
21. Issues  with  terms   Some  of  which  got  resolved  at  this  mee5ng  
22. How  to  populate  terms?   Defini5ons  ≠  guidelines  (by  design)  
23. taxonID    scien5ficNameID      acceptedNameUsageID        parentNameUsageID          originalNameUsageID            nameAccordingToID              namePublishedInID                taxonConceptID                acceptedNameUsage                  parentNameUsage                    originalNameUsage                      nameAccordingTo                        namePublishedIn                          namePublishedInYear   Too  many  op5ons   E.g.  taxon  terms,  ID  vs  code  
24. Specific  uses  cases   We  need  guidelines!  
25. Metadata  issues  
26. What  is  important?   IPT  vs  GBIF  page  vs  data  paper  
27. Abstract   Contact   Associa5on   Coverage   Methodology   References   Rights/Purpose/Add.   Project   Download   Links   Cita5on   Coverage   Keywords   Associa5on   Project   Methodology   Cita5on   References   Purpose/Rights/Add.   Links   Contact   Abstract   Keywords   IPT  metadata  editor   IPT  resource  page  
28. Coverage   Keywords   Associa5on   Project   Methodology   Cita5on   References   Purpose/Rights/Add.   Links   Contact   Abstract   Coverage   Abstract   Purpose   Addi5onal   Contact   Project   Methodology   Assoc.   References   Cita5on/Rights   ? Links   Download   IPT  metadata  editor   GBIF  resource  page  
29. Coverage   Keywords   Associa5on   Project   Methodology   Cita5on   References   Purpose/Rights/Add.   Links   Contact   Abstract   Abstract   Keywords   Download   Purpose   Coverage   Technical/Rights   Addi5onal   Methodology   Project   Associa5on   References   Contact/Cita5on   Authors   IPT  metadata  editor   Data  paper  
30. Too  many  op5ons   Creator  /  Maintainer  
31. People   Basic  metadata,  associated     par5es,  project  personnel  
32. Study  area  descrip5on,  design     descrip5on,  study  area  extent  
33. We’d  also  like  to  add   Technical  info,  publica5on  interval,  norms  
34. Are  we  hiing  the  limita5ons  of   EML?  
35. Rethink  and  document   We  need  guidelines!  
36. Ways  forward  
37. Collabora5ve  guidelines   Contribute,  fork  &  merge  them  on  GitHub  
38. github.com/LifeWatchINBO/data-­‐publica5on-­‐guidelines  
39. github.com/tdwg/dwc-­‐guidelines  
40. Bejer  integra5on   Between  GitHub  &  IPT  
41. Plug  &  Play  cleaning   APIs  via  Google  Refine  extensions  
42. Collabora5ve  tagging   This  dataset  is  useful  for  this  
43. ckan.org  
44. cer5ficates.theodi.org  
45. Feedback  from  users   Are  we  on  the  right  track?  
46. Thanks!   @peterdesmet   @dimibro   @LifeWatchINBO