depsolver (pycon.fr)

A presentation about depsolver, a new pure-python library for dependencies handling,

Transcript

1. DEPSOLVER
   A tool for dependency solving
   @cournape
   github.com/cournape
   Monday, 28 October 13
   

   View Slide

2. CURRENT SITUATION
   • setuptools: implements dependency handling (install_requires)
   • pip: another implementation (requirements.txt)
   Monday, 28 October 13
   

   View Slide

3. EXAMPLE
   # - A requires B and C
   # - B requires D <= 1.1
   # - C requires D <= 0.9
   $  pip  install
   #  installs  D  1.1
   Monday, 28 October 13
   

   View Slide

4. LINUX DISTRIBUTIONS
   • Successful examples of distributed packaging:
   • apt-get
   • urpmi/smart/yum
   • etc...
   • Graph-based algorithms with complex rules
   • I don’t want to re-implement apt-get...
   Monday, 28 October 13
   

   View Slide

5. REQUIREMENTS
   • pure python
   • works for python 2 and 3
   • flexible version representation (use @enthought)
   Monday, 28 October 13
   

   View Slide

6. Monday, 28 October 13
   

   View Slide

7. PHP TO THE RESCUE
   • PHP Composer: php package manager
   • https://github.com/composer/composer
   • Uses SAT solver for dependencies
   • Open source, well written code, small:
   • 15k LOC, only 2.5K LOC for dependency handling
   Monday, 28 October 13
   

   View Slide

8. SAT SOLVER ?
   • SATisfiability problem: “decision problem with boolean
   expressions”
   • NP-complete (~ brute force-only solutions)
   Monday, 28 October 13
   

   View Slide

9. EXAMPLES
   • A & (B | C) is satisfiable (e.g. A = B = True, C = False)
   • A & ( B | C) & ~B & ~C is not satisfiable
   Monday, 28 October 13
   

   View Slide

10. FROM DEPENDENCIES
    TO SAT
    Monday, 28 October 13
    

    View Slide

11. FROM SAT TO DEPENDENCIES
    • Installing package “A”: (A)
    • Removing “A”: (~A)
    • “A” dependencies: (~A | Dep-v1 | Dep-v2 | ...)
    • Example: A depends on B >= 1.2.2, B-1.3.0 and B-1.4.0 are
    available becomes (~A | B-1.3.0 | B-1.4.0)
    Monday, 28 October 13
    

    View Slide

12. FROM SAT TO DEPENDENCIES
    • A conflicts with B >= 1.2: (~A | ~B-1.2) & (~A | ~B-1.3)
    • A obsoletes B >= 1.2: same as conflict
    Monday, 28 October 13
    

    View Slide

13. SOLVER ALGORITHM
    • minisat, etc... BSD implementations available:
    • Simple solvers may be implemented in a few 10s of lines
    • Davis–Putnam–Logemann–Loveland (DPLL) algorithm is a
    complete, backtracking-based search algorithm
    Monday, 28 October 13
    

    View Slide

14. DEPSOLVER
    Monday, 28 October 13
    

    View Slide

15. DEPSOLVER
    • Port of composer solver in python
    • Small (~ 3kloc)
    • Simple (but low-level) API
    Monday, 28 October 13
    

    View Slide

16. EXAMPLE
    remote_repository = Repository([
    P("A-1.0.0; depends (B, C)"),
    P("B-1.0.0; depends (D <= 1.1.0)"),
    P("C-1.0.0; depends (D <= 0.9.0)"),
    P("D-1.1.0"),
    P("D-0.9.0")])
    installed_repository = Repository()
    pool = Pool([remote_repository, installed_repository])
    request = Request(pool)
    request.install(Requirement.from_string("A"))
    for operation in Solver(pool, installed_repository).solve(request):
    print operation
    Monday, 28 October 13
    

    View Slide

17. MAIN CONCEPTS
    • PackageInfo: package metadata
    • Repository: bag of packages
    • Pool: bag of repositories (retrieve packages following a
    requirement)
    • Solver: actual solver
    Monday, 28 October 13
    

    View Slide

18. FLEXIBLE VERSION SUPPORT
    • PEP 440 supported
    • other (better) format also supported: SEMVER, Debian
    Monday, 28 October 13
    

    View Slide

19. EXAMPLE
    from depsolver.debian_version import DebianVersion
    P = PackageInfo.from_string
    V = DebianVersion.from_string
    a_1_0_0 = P("A-1.0.0~1; depends (B, C)", V)
    b_1_0_0 = P("B-1.0.0~1; depends (D <= 1.1.0~1)", V)
    c_1_0_0 = P("C-1.0.0~1; depends (D <= 0.9.0~1)", V)
    d_1_1_0 = P("D-1.1.0~1", V)
    d_0_9_0 = P("D-0.9.0~1", V)
    Monday, 28 October 13
    

    View Slide

20. FUTURE PLAN
    • Add support for removal and updates
    • Finish support for yaml-based scenario description
    • Proof of concept for integration with pip
    Monday, 28 October 13
    

    View Slide

21. WHERE TO GET IT
    • https://github.com/enthought/depsolver
    Monday, 28 October 13
    

    View Slide