depsolver (pycon.fr)

A presentation about depsolver, a new pure-python library for dependencies handling,

Transcript

1. DEPSOLVER A tool for dependency solving @cournape github.com/cournape Monday, 28 October 13
2. CURRENT SITUATION • setuptools: implements dependency handling (install_requires) • pip: another implementation (requirements.txt) Monday, 28 October 13
3. EXAMPLE # - A requires B and C # - B requires D <= 1.1 # - C requires D <= 0.9 $  pip  install #  installs  D  1.1 Monday, 28 October 13
4. LINUX DISTRIBUTIONS • Successful examples of distributed packaging: • apt-get • urpmi/smart/yum • etc... • Graph-based algorithms with complex rules • I don’t want to re-implement apt-get... Monday, 28 October 13
5. REQUIREMENTS • pure python • works for python 2 and 3 • flexible version representation (use @enthought) Monday, 28 October 13
6. Monday, 28 October 13
7. PHP TO THE RESCUE • PHP Composer: php package manager • https://github.com/composer/composer • Uses SAT solver for dependencies • Open source, well written code, small: • 15k LOC, only 2.5K LOC for dependency handling Monday, 28 October 13
8. SAT SOLVER ? • SATisfiability problem: “decision problem with boolean expressions” • NP-complete (~ brute force-only solutions) Monday, 28 October 13
9. EXAMPLES • A & (B | C) is satisfiable (e.g. A = B = True, C = False) • A & ( B | C) & ~B & ~C is not satisfiable Monday, 28 October 13
10. FROM DEPENDENCIES TO SAT Monday, 28 October 13
11. FROM SAT TO DEPENDENCIES • Installing package “A”: (A) • Removing “A”: (~A) • “A” dependencies: (~A | Dep-v1 | Dep-v2 | ...) • Example: A depends on B >= 1.2.2, B-1.3.0 and B-1.4.0 are available becomes (~A | B-1.3.0 | B-1.4.0) Monday, 28 October 13
12. FROM SAT TO DEPENDENCIES • A conflicts with B >= 1.2: (~A | ~B-1.2) & (~A | ~B-1.3) • A obsoletes B >= 1.2: same as conflict Monday, 28 October 13
13. SOLVER ALGORITHM • minisat, etc... BSD implementations available: • Simple solvers may be implemented in a few 10s of lines • Davis–Putnam–Logemann–Loveland (DPLL) algorithm is a complete, backtracking-based search algorithm Monday, 28 October 13
14. DEPSOLVER Monday, 28 October 13
15. DEPSOLVER • Port of composer solver in python • Small (~ 3kloc) • Simple (but low-level) API Monday, 28 October 13
16. EXAMPLE remote_repository = Repository([ P("A-1.0.0; depends (B, C)"), P("B-1.0.0; depends (D <= 1.1.0)"), P("C-1.0.0; depends (D <= 0.9.0)"), P("D-1.1.0"), P("D-0.9.0")]) installed_repository = Repository() pool = Pool([remote_repository, installed_repository]) request = Request(pool) request.install(Requirement.from_string("A")) for operation in Solver(pool, installed_repository).solve(request): print operation Monday, 28 October 13
17. MAIN CONCEPTS • PackageInfo: package metadata • Repository: bag of packages • Pool: bag of repositories (retrieve packages following a requirement) • Solver: actual solver Monday, 28 October 13
18. FLEXIBLE VERSION SUPPORT • PEP 440 supported • other (better) format also supported: SEMVER, Debian Monday, 28 October 13
19. EXAMPLE from depsolver.debian_version import DebianVersion P = PackageInfo.from_string V = DebianVersion.from_string a_1_0_0 = P("A-1.0.0~1; depends (B, C)", V) b_1_0_0 = P("B-1.0.0~1; depends (D <= 1.1.0~1)", V) c_1_0_0 = P("C-1.0.0~1; depends (D <= 0.9.0~1)", V) d_1_1_0 = P("D-1.1.0~1", V) d_0_9_0 = P("D-0.9.0~1", V) Monday, 28 October 13
20. FUTURE PLAN • Add support for removal and updates • Finish support for yaml-based scenario description • Proof of concept for integration with pip Monday, 28 October 13
21. WHERE TO GET IT • https://github.com/enthought/depsolver Monday, 28 October 13