Python Packaging in Fedora

For Flock (Fedora contributors' conference) in Krakow

Transcript

1. Python Packaging in Fedora
   Petr Viktorin
   pviktori@redhat.com
   Flock, 2016-08-02
   

   View Slide

2. View Slide

3. I. Python 3
   

   View Slide

4. Python 1991
   Java 1995
   

   View Slide

5. Python 1991
   Java 1995
   Python 3.0 2008
   Node.js 2009
   

   View Slide

6. Python 1991
   Java 1995
   Python 3.0 2008
   Node.js 2009
   Python 3.5 2015
   

   View Slide

7. Python 1991
   Java 1995
   Python 3.0 2008
   Node.js 2009
   Python 3.5 2015
   Python 2 EOL 2020
   

   View Slide

8. Python 3 Porting Database
   http://fedora.portingdb.xyz
   

   View Slide

9. View Slide

10. at least
    49,9%
    of Python software
    packaged in Fedora
    supports Python 3
    

    View Slide

11. at least
    49,9%
    of Python software
    packaged in Fedora
    supports Python 3
    45,5%
    has the Py 3 version packaged
    

    View Slide

12. Help us port!
    3
    PYTHON3
    PYTHON3
    PYTHON
    http://fedora.portingdb.xyz
    

    View Slide

13. View Slide

14. Fedora infra
    Desktop toolkits
    Big non-Python apps
    

    View Slide

15. I. Python 3
    II. Automating
    Packaging
    

    View Slide

16. A packager should:
    1. Make sure SW plays nice
    2. Integrate with rest of system
    3. Check licenses, patents, etc.
    

    View Slide

17. A packager should:
    1. Make sure SW plays nice
    2. Integrate with rest of system
    3. Check licenses, patents, etc.
    4. Read 50 pages of guidelines
    5. Know a weird macro language
    

    View Slide

18. %global pypi_name mypkg
    Name: python -%{pypi_name}
    Version: 2.4.25.1
    Release: 1%{?dist}
    License: Python
    URL: https://github.com
    /%{pypi_name}/%{pypi_name}
    Source0: https://pypi.python
    .org/packages/source/p/%{
    pypi_name}/%{pypi_name}-%{version
    }.tar.gz
    

    View Slide

19. %if 0%{?rhel} && 0%{?rhel} <= 6
    %{!?__python2: %global \
    __python2 /usr/bin/python2}
    %{!?python2_sitelib: %global \
    python2_sitelib %(%{__python2} -c
    %endif
    %if 0%{?fedora} > 12 || 0%{?rhel} >
    %global with_python3 1
    %global __python3 python3
    %endif
    

    View Slide

20. from distutils.core import setup
    setup(
    name='mypkg ',
    version='2.4.25.1',
    author='Me',
    url='github.com/me/mypkg/',
    packages=['mypkg '],
    install_requires=['six '],
    )
    

    View Slide

21. Enter pyp2rpm
    # dnf install /usr/bin/pyp2rpm
    $ pyp2rpm mypkg
    

    View Slide

22. Enter pyp2rpm
    # dnf install /usr/bin/pyp2rpm
    $ pyp2rpm mypkg
    It might actually work!
    

    View Slide

23. We can use pyp2rpm to...
    Auto-build all of PyPI in COPR!
    

    View Slide

24. We can use pyp2rpm to...
    Auto-build all of PyPI in COPR!
    Why?
    

    View Slide

25. We can use pyp2rpm to...
    Auto-build all of PyPI in COPR!
    Why?
    - Test pyp2rpm
    

    View Slide

26. We can use pyp2rpm to...
    Auto-build all of PyPI in COPR!
    Why?
    - Test pyp2rpm
    - Run upstream tests
    

    View Slide

27. We can use pyp2rpm to...
    Auto-build all of PyPI in COPR!
    Why?
    - Test pyp2rpm
    - Run upstream tests
    - Provide a repository
    

    View Slide

28. dnf pip install -r requirements.txt
    (hypothetical command)
    

    View Slide

29. “PyPI” name Fedora pkg
    pyopencl python3-pyopencl
    mypy-lang python3-mypy
    

    View Slide

30. “PyPI” name Fedora pkg
    pyopencl python3-pyopencl
    mypy-lang python3-mypy
    $ dnf repoquery --provides
    python3 -pyopencl
    ...
    python3.5dist(pyopencl) = 2015.2
    ...
    

    View Slide

31. “PyPI” name Fedora pkg
    pyopencl python3-pyopencl
    mypy-lang python3-mypy
    $ dnf repoquery --provides
    python3 -pyopencl
    ...
    python3.5dist(pyopencl) = 2015.2
    ...
    Live in Fedora 25!
    

    View Slide

32. I. Python 3
    II. Automating Packaging
    III. System
    Python
    

    View Slide

33. System Python
    /usr/libexec/system-python
    An effort to minimize minimal
    installs
    (cloud images)
    

    View Slide

34. Python stdlib by disk size
    test
    asyncio
    email
    tkinter
    encodings
    distutils
    unittest
    pydoc_data
    ensurepip
    ctypes
    lib2to3
    idlelib
    individual
    modules
    

    View Slide

35. System Python libs
    /usr/libexec/system-python
    /usr/lib/python3.5/
    lib2to3
    unittest
    venv
    ensurepip
    pydoc
    encodings
    sqlite3
    collections
    logging
    distutils
    html
    test
    tkinter
    turtledemo
    idlelib
    system-python-libs
    python3-libs
    python3-test
    python3-tkinter
    python3
    system-python
    /usr/bin/python3
    

    View Slide

36. Possible users: DNF, cloud-init
    

    View Slide

37. Possible users: DNF, cloud-init
    Future: Isolation of system
    tools?
    

    View Slide

38. I. Python 3
    II. Automating Packaging
    III. System Python
    IV. Python 3.6
    

    View Slide

39. Python 3.6
    * Format Strings
    * Large Speedups
    * Advanced class creation tools
    

    View Slide

40. Python 3.6 Alpha 1 2016-05-17
    Fedora 25 Alpha Freeze 2016-08-09
    

    View Slide

41. Python 3.6 Alpha 1 2016-05-17
    Fedora 25 Alpha Freeze 2016-08-09
    Python 3.6 Beta 1 2016-09-12
    Fedora 25 Beta Freeze 2016-09-20
    

    View Slide

42. Python 3.6 Alpha 1 2016-05-17
    Fedora 25 Alpha Freeze 2016-08-09
    Python 3.6 Beta 1 2016-09-12
    Fedora 25 Beta Freeze 2016-09-20
    Python 3.6 RC 1 2016-12-05
    Fedora 25 Final Freeze 2016-10-25
    

    View Slide

43. Python 3.6 Alpha 1 2016-05-17
    Fedora 25 Alpha Freeze 2016-08-09
    Python 3.6 Beta 1 2016-09-12
    Fedora 25 Beta Freeze 2016-09-20
    Python 3.6 RC 1 2016-12-05
    Fedora 25 Final Freeze 2016-10-25
    Python 3.6 Final 2016-12-16
    Fedora 26 Branches 2016-12-???
    

    View Slide

44. I. Python 3
    II. Automating Packaging
    III. System Python
    IV. Python 3.6
    V. Who's “we”?
    

    View Slide

45. python-maint @ Red Hat
    

    View Slide

46. python-maint @ Red Hat
    

    View Slide

47. python-maint @ Red Hat
    

    View Slide

48. python-maint @ Red Hat
    

    View Slide

49. python-maint @ Red Hat
    

    View Slide

50. python-maint @ Red Hat
    

    View Slide

51. Sorry!
    We'll do better!
    

    View Slide

52. We're Python SIG
    python-devel
    @lists.fedoraproject.org
    #fedora-python on Freenode
    fedora-python on Github
    

    View Slide

53. I. Python 3
    II. Automating Packaging
    III. System Python
    IV. Python 3.6
    V. Who's “we”?
    

    View Slide