Upgrade to Pro — share decks privately, control downloads, hide ads and more …

depsolver (pycon.fr)

cournape
October 27, 2013

depsolver (pycon.fr)

A presentation about depsolver, a new pure-python library for dependencies handling,

cournape

October 27, 2013
Tweet

More Decks by cournape

Other Decks in Programming

Transcript

  1. DEPSOLVER
    A tool for dependency solving
    @cournape
    github.com/cournape
    Monday, 28 October 13

    View Slide

  2. CURRENT SITUATION
    • setuptools: implements dependency handling (install_requires)
    • pip: another implementation (requirements.txt)
    Monday, 28 October 13

    View Slide

  3. EXAMPLE
    # - A requires B and C
    # - B requires D <= 1.1
    # - C requires D <= 0.9
    $  pip  install
    #  installs  D  1.1
    Monday, 28 October 13

    View Slide

  4. LINUX DISTRIBUTIONS
    • Successful examples of distributed packaging:
    • apt-get
    • urpmi/smart/yum
    • etc...
    • Graph-based algorithms with complex rules
    • I don’t want to re-implement apt-get...
    Monday, 28 October 13

    View Slide

  5. REQUIREMENTS
    • pure python
    • works for python 2 and 3
    • flexible version representation (use @enthought)
    Monday, 28 October 13

    View Slide

  6. Monday, 28 October 13

    View Slide

  7. PHP TO THE RESCUE
    • PHP Composer: php package manager
    • https://github.com/composer/composer
    • Uses SAT solver for dependencies
    • Open source, well written code, small:
    • 15k LOC, only 2.5K LOC for dependency handling
    Monday, 28 October 13

    View Slide

  8. SAT SOLVER ?
    • SATisfiability problem: “decision problem with boolean
    expressions”
    • NP-complete (~ brute force-only solutions)
    Monday, 28 October 13

    View Slide

  9. EXAMPLES
    • A & (B | C) is satisfiable (e.g. A = B = True, C = False)
    • A & ( B | C) & ~B & ~C is not satisfiable
    Monday, 28 October 13

    View Slide

  10. FROM DEPENDENCIES
    TO SAT
    Monday, 28 October 13

    View Slide

  11. FROM SAT TO DEPENDENCIES
    • Installing package “A”: (A)
    • Removing “A”: (~A)
    • “A” dependencies: (~A | Dep-v1 | Dep-v2 | ...)
    • Example: A depends on B >= 1.2.2, B-1.3.0 and B-1.4.0 are
    available becomes (~A | B-1.3.0 | B-1.4.0)
    Monday, 28 October 13

    View Slide

  12. FROM SAT TO DEPENDENCIES
    • A conflicts with B >= 1.2: (~A | ~B-1.2) & (~A | ~B-1.3)
    • A obsoletes B >= 1.2: same as conflict
    Monday, 28 October 13

    View Slide

  13. SOLVER ALGORITHM
    • minisat, etc... BSD implementations available:
    • Simple solvers may be implemented in a few 10s of lines
    • Davis–Putnam–Logemann–Loveland (DPLL) algorithm is a
    complete, backtracking-based search algorithm
    Monday, 28 October 13

    View Slide

  14. DEPSOLVER
    Monday, 28 October 13

    View Slide

  15. DEPSOLVER
    • Port of composer solver in python
    • Small (~ 3kloc)
    • Simple (but low-level) API
    Monday, 28 October 13

    View Slide

  16. EXAMPLE
    remote_repository = Repository([
    P("A-1.0.0; depends (B, C)"),
    P("B-1.0.0; depends (D <= 1.1.0)"),
    P("C-1.0.0; depends (D <= 0.9.0)"),
    P("D-1.1.0"),
    P("D-0.9.0")])
    installed_repository = Repository()
    pool = Pool([remote_repository, installed_repository])
    request = Request(pool)
    request.install(Requirement.from_string("A"))
    for operation in Solver(pool, installed_repository).solve(request):
    print operation
    Monday, 28 October 13

    View Slide

  17. MAIN CONCEPTS
    • PackageInfo: package metadata
    • Repository: bag of packages
    • Pool: bag of repositories (retrieve packages following a
    requirement)
    • Solver: actual solver
    Monday, 28 October 13

    View Slide

  18. FLEXIBLE VERSION SUPPORT
    • PEP 440 supported
    • other (better) format also supported: SEMVER, Debian
    Monday, 28 October 13

    View Slide

  19. EXAMPLE
    from depsolver.debian_version import DebianVersion
    P = PackageInfo.from_string
    V = DebianVersion.from_string
    a_1_0_0 = P("A-1.0.0~1; depends (B, C)", V)
    b_1_0_0 = P("B-1.0.0~1; depends (D <= 1.1.0~1)", V)
    c_1_0_0 = P("C-1.0.0~1; depends (D <= 0.9.0~1)", V)
    d_1_1_0 = P("D-1.1.0~1", V)
    d_0_9_0 = P("D-0.9.0~1", V)
    Monday, 28 October 13

    View Slide

  20. FUTURE PLAN
    • Add support for removal and updates
    • Finish support for yaml-based scenario description
    • Proof of concept for integration with pip
    Monday, 28 October 13

    View Slide

  21. WHERE TO GET IT
    • https://github.com/enthought/depsolver
    Monday, 28 October 13

    View Slide