Save 37% off PRO during our Black Friday Sale! »

Whip: Communicate and test what to expect from data

Whip: Communicate and test what to expect from data

Talk at the TDWG 2018 annual conference in Dunedin, New Zealand - August 28, 2018.

Abstract: https://doi.org/10.3897/biss.2.25317

6f6914b1cdb438695ec1aaabba7463bb?s=128

Peter Desmet

August 28, 2018
Tweet

Transcript

  1. Whip Communicate and test what to expect from data Stijn

    Van Hoey & Peter Desmet
  2. Expectations Data Users

  3. Expectations Data Users Fit for my research? Fit for specific

    user community?
  4. We are a data publisher

  5. We care

  6. What to expect Data Publisher

  7. What to expect Data Publisher Data quality Standardization Community recommendations

    Dataset characteristics
  8. Expectations / What to expect Data Publisher Users Expectations What

    to expect
  9. How to communicate expectations? Data Publisher Users Expectations What to

    expect
  10. How to test expectations? Data Publisher Users Expectations What to

    expect
  11. Whip

  12. Whip syntax

  13. Whip syntax

  14. Whip syntax Field

  15. Whip syntax Field Specification

  16. Whip syntax Comment Field Specification

  17. Whip syntax Comment Field Specification

  18. Whip specifications allowed minlength / maxlength stringformat regex min /

    max numberformat mindate / maxdate dateformat
  19. Whip scope specifications empty delimitedvalues if

  20. Using whip to document

  21. Pywhip: a whip implementation

  22. import whip_csv from pywhip # load specifications with open("my_specifications.yml") as

    spec_file: specifications = yaml.load(spec_file) # test specifications test = whip_csv("my_data.csv", specifications) # get report test.get_report("html") Pywhip Or “json”
  23. Pywhip

  24. Pywhip

  25. Pywhip

  26. Conclusion Human and machine-readable syntax to express specifications for data

    Not specific to Darwin Core (but we plan to use it for that) Can be adopted by users (expectations) and publishers (what to expect) Can be included with dataset as testable metadata Pywhip: first implementation for testing whip specifications
  27. github.com/inbo/whip github.com/inbo/pywhip bit.ly/pywhip_binder Thank you! Data Specifications