Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Towards "annex", a Fact Based Dependency System

Mark Hibberd
September 19, 2014

Towards "annex", a Fact Based Dependency System

Knowledge is not static. Yet when dealing with program artifacts, we choose to seal off what we know at the point in time when we know the least. This is wrong.

Context is important. Yet when defining dependencies on artifacts, instead of directly specify the query we want (and hence embedding its context), we manually translate our request into antiquated notions of meta-data, encoded as a number, embedded in a string. Yes, semantic versioning is wrong.

Reproducibility is essential. Yet most existing dependency systems force a trade off of rigour and reproducibility against flexibility and ease of use. This is not necessary.

Drawing on well understood foundations from datalog and deductive databases, and utilizing functional programming fundamentals, “annex” takes a different view on how to manage artifacts. We should be able to ask: “Give me the latest binary compatible versions of X with no known CVE”; or, “Give me the last stable builds of my dependencies that have been tested in IE 9, Chrome and Firefox”; or in a more general context outside of dependency resolution, queries such as “What platforms has build x of my library been tested on?” provide a useful understanding of the current state of artifacts; and finally, it should be possible to phrase all of these questions with a first class notion of time, for example “Give me the same dependencies when I last asked this query”.

This talk will start by walking through the concepts behind “annex”, before taking a deeper look at the design and implementation (in Haskell) and a multi-language demonstration. We will look at how its functional underpinnings give rise to very desirable properties for a dependency system. These properties include: trivial distribution and caching; guaranteed reproducibility with minimal context; predictable performance; and interestingly, how steadfastly holding to functional programming principles contributes to being able to deliver a humane user experience in the face of complexity.

Presented at strangeloop, 19th of September 2014 - https://thestrangeloop.com/sessions/towards-annex-a-fact-based-dependency-system

Mark Hibberd

September 19, 2014

More Decks by Mark Hibberd

Other Decks in Programming


  1. “The enemy is the gramophone mind, whether or not one

    agrees with the record that is being played at the moment.” George Orwell - The Freedom of the Press
  2. ci

  3. cve

  4. @v1 boxer 1.2.1 commit: bd2f074…02 api-signature: […] feature: it-works @v2

    boxer 1.2.1 commit: bd2f074…02 api-signature: […] feature: it-works tested: FreeBSD-9.1
  5. :boxer :has :feature “multi-part-put” :has :commit “abcd-1345” :has :branch “master”

    ! :snowball :is :compatable-with atom/… ! :napoleon :semver >= 1.3 < 1.4 !
  6. :boxer :has :feature “multi-part-put” :has :commit “abcd-1345” :has :branch “master”

    ! :snowball :is :compatable-with atom/… ! :napoleon :semver >= 1.3 < 1.4 ! +annex.example.com@v12345
  7. :boxer :has :feature “multi-part-put” :has :commit “abcd-1345” :has :branch “master”

    ! :snowball :is :compatable-with atom/… ! :napoleon :semver >= 1.3 < 1.4 +annex.internal.com@v123
  8. :boxer :has :feature “multi-part-put” :has :commit “abcd-1345” :has :branch “master”

    ! :snowball :is :compatable-with atom/… ! :napoleon :semver >= 1.3 < 1.4 :no-cve +annex.example.com@HEAD
  9. git checkout -b topic/feature git add src/Boxer.hs git commit -m

    ‘Great change! fixes #12 annex: :feature winning’ git push origin topic/feature annex fact atom/12…ef --git HEAD
  10. git notes add --ref=annex \ -m “:feature again” HEAD git

    push origin refs/notes/* annex fact atom/12…ef --git HEAD
  11. Usability #9 Start with the premise that you need to

    interact with less principled systems
  12. :boxer :has :feature ingestion ! :snowball :has :feature timeline !

    :eminence :has :commit ab34…f3e1 :transitive napoleon.ax:
  13. :boxer :has :feature ingestion ! :snowball :has :feature timeline !

    :eminence :has :fix #112 :transitive napoleon.ax:
  14. :boxer :has :feature ingestion ! :snowball :has :feature timeline !

    :annex-resolver :has :version 1 napoleon.ax:
  15. 0 25 50 75 100 April May June July Deeper

    Analytics arbitrary queries and reporting over atoms
  16. 0 25 50 75 100 April May June July Deeper

    Analytics inference of relevant facts for customer issues
  17. 0 25 50 75 100 April May June July Deeper

    Analytics predict failure in advance
  18. Images ! Unmodified, Licences Specified at WikiMedia links" http://commons.wikimedia.org/wiki/File:Clock_Gare_de_Paris-Est.jpg! http://en.wikipedia.org/wiki/File:Bundesarchiv_Bild_101III-

    Merz-014-12A,_Russland,_Beginn_Unternehmen_Zitadelle,_Panzer.jpg! http://commons.wikimedia.org/wiki/File:French_-_Door_with_Cat_Hole_- _Walters_64164.jpg! ! Unmodified. CC BY 2.0 (https://creativecommons.org/licenses/by/2.0/)" https://www.flickr.com/photos/timothymorgan/75288582/! https://www.flickr.com/photos/timothymorgan/75288583/! https://www.flickr.com/photos/timothymorgan/75294154/! https://www.flickr.com/photos/timothymorgan/75593155/! https://www.flickr.com/photos/timothymorgan/75593155/