Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Robert Collins: Semver and Python with PBR

Robert Collins: Semver and Python with PBR

= = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = =
Robert Collins:
Semver and Python with PBR
= = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = =
@ Kiwi PyCon 2014 - Saturday, 13 Sep 2014 - Track 1
http://kiwi.pycon.org/

**Audience level**

Novice

**Description**

PBR is a setuptools plugin which OpenStack developed to provide simple and consistent minimal-boilerplate build definitions for its projects. Semantic versioning provides simple and robust rules for deciding on version numbers, and PBR will soon be able to manage the entire process of version number assignment in a project... and more.

**Abstract**

PBR is a setuptools plugin which OpenStack developed to provide simple and consistent minimal-boilerplate build definitions for its projects. Now used by all the OpenStack projects, PBR provides integration glue for core features:

- testing
- binary package creation for Linux distributors
- inclusion of files in tarballs
- changelog and authors file creation
- pypi summary creation
- version number creation
- sphinx doc stub creation and manpage enablement
- unified requirements management
- for both easy-install and pip with single-file control

The most interesting part is the version number creation, since coming up with the right version number can be a contentious discussion in some projects. Semver provides simple and robust rules for deciding on version numbers, and I'm in the middle of implementing automation for these in PBR itself, with integration glue to export them in PEP-440, dpkg and rpm format. The only dependencies PBR has are git + a recent pip, so this should be useful for many attendees - and while PBR is an OpenStack invention we're very interested in making sure its useful and reliable for anyone that wants to use it.

**YouTube**

https://www.youtube.com/watch?v=6hjg-Uia5AU

6b880a0b67fac54c42c77fe70d97334d?s=128

New Zealand Python User Group

September 13, 2014
Tweet

Transcript

  1. pbr and semver aka Python Build Reasonableness and Semantic Versioning

    Robert Collins rbtcollins@hp.com @rbtcollins (Twitter)
  2. Setuptools

  3. OpenStack Dozens of projects Identical infrastructure code Maintenance nightmare

  4. Refactor New project Setuptools plugin Used by all OpenStack projects

  5. PBR Consistent setuptools integration glue for all OpenStack projects. Depends

    on git and a recent pip for installation support.
  6. Getting started with PBR

  7. setup.py # ! / u s r / b i

    n / e n v p y t h o n i m p o r t s e t u p t o o l s s e t u p t o o l s . s e t u p ( s e t u p _ r e q u i r e s = [ ' p b r ' ] , p b r = T r u e )
  8. setup.cfg [ m e t a d a t a

    ] n a m e = o s - a p p l y - c o n f i g a u t h o r = O p e n S t a c k a u t h o r - e m a i l = o p e n s t a c k - d e v @ l i s t s . o p e n s t a c k . o r g s u m m a r y = C o n f i g f i l e s f r o m c l o u d m e t a d a t a d e s c r i p t i o n - f i l e = R E A D M E . r s t h o m e - p a g e = h t t p : / / g i t . o p e n s t a c k . o r g / c g i t / o p e n s t a c k / o s - a p p l y - c o n f i g c l a s s i f i e r = D e v e l o p m e n t S t a t u s : : 4 - B e t a . . . . P r o g r a m m i n g L a n g u a g e : : P y t h o n [ f i l e s ] p a c k a g e s = o s _ a p p l y _ c o n f i g [ e n t r y _ p o i n t s ] c o n s o l e _ s c r i p t s = o s - a p p l y - c o n f i g = o s _ a p p l y _ c o n f i g . a p p l y _ c o n f i g : m a i n
  9. PBR Features

  10. Testing entrypoint integrated p y t h o n s

    e t u p . p y t e s t
  11. git integration Hard requirement but.... automatically picks up all versioned

    files scans history for changelog and authors file contents
  12. PyPI integration Generates a rich summary from README.rst + changelog

  13. Documentation Sphinx API glue, and Sphinx manpage output.

  14. PIP Pulls requirements from requirements*.txt

  15. Versioning Generates pre-release version numbers for you from tags and

    or config settings
  16. Semantic versioning is a simple and clear set of rules

    for versioning software to clearly indicate compatibility interactions. semver
  17. openstack-semver "This is a fork of Semantic Versioning 2.0. The

    specific changes have to do with the format of pre-release and build labels, specifically to make them not confusing when co-existing with Linux Distribution packaging. Inspiration for the format of the pre-release and build labels came from Python's PEP440."
  18. Issues

  19. dev builds 1.3.2 tagged 5 local commits 1.3.2.5.g$sha

  20. pbr-semver Developers don't know the semver rules we've settled on:

    tag 1.3.2.a2 should be 1.3.2.0a2
  21. CD 1.3.2.0a1 cannot be sorted vs 1.3.2.dev5

  22. Binary Packaging Turns out everyone reinvents the same integration and

    version translation glue... differently.
  23. Sort-before 1.3.2.0a1 sorts before 1.3.2 - versions are special. dpkg

    has ~ rpm has nothing
  24. pbr-semver new shiny magic

  25. Strict pbr-semver mode Enforce pbr-semver compatible version numbers

  26. Epic fail on existing versions

  27. Have to parse every version we ever accepted: 1 .

    2 . 1 a 1 1 . 2 . 3 . 4 . g 1 2 3 1 . 2 . b 1 1 . 2 b 1 2 . 1 . 0 . r c 1
  28. History scanning Scan git history for pseudo headers and calculate

    new release versions
  29. Before: pbr-0.10.1.dev5.gaada1d1

  30. L o o k f o r a n d

    p r o c e s s s e m - v e r p s e u d o h e a d e r s i n g i t A t t h e m o m e n t c a r e f u l c h a n g e l o g r e v i e w i s n e e d e d b y h u m a n d s t o d e t e r m i n e t h e n e x t v e r s i o n n u m b e r . S e m a n t i c v e r s i o n i n g m e a n s t h a t a l l w e n e e d t o k n o w t o g e t a r e a s o n a b l e d e f a u l t n e x t v e r s i o n ( a n d a l o w e r b o u n d o n t h e n e x t v e r s i o n ) i s a r e c o r d o f b u g f i x / d e p r e c a t i o n / f e a t u r e w o r k a n d a p i - b r e a k i n g c o m m i t s . I n t h i s p a t c h w e s t a r t s c a n n i n g f o r s u c h h e a d e r s f r o m t h e g i t h i s t o r y t o r e m o v e t h i s b u r d e n f r o m d e v e l o p e r s / p r o j e c t r e l e a s e m a n a g e r s . H i g h e r v e r s i o n s c a n o f c o u r s e b e u s e d e i t h e r v i a p r e - v e r s i o n i n g o r b y t a g g i n g t h e d e s i r e d v e r s i o n . i m p l e m e n t s : b l u e p r i n t p b r - s e m v e r s e m - v e r : f e a t u r e < - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - C h a n g e - I d : I d 5 e 8 c d 7 2 3 d 5 1 8 6 d 1 b d 8 c 0 1 5 9 9 e a e 8 9 3 3 e 6 f 7 e a 6 d
  31. After: pbr-0.11.0.dev5.gaada1d1

  32. s e m - v e r : f e

    a t u r e s e m - v e r : a p i - b r e a k s e m - v e r : d e p r e c a t i o n s e m - v e r : b u g f i x s e m - v e r : b u g f i x , a p i - b r e a k
  33. (planned) Doing releases W#rap git tag to make releases trivial

    p y t h o n s e t u p . p y t a g - r e l e a s e
  34. (planned) Doing binary (distro) builds Provide helpers to generate dpkg

    and RPM versions from a semver version s e t u p . p y d e b - v e r s i o n 1 . 2 . 0 ~ 0 b 1 s e t u p . p y r p m - v e r s i o n 1 . 1 . 9 9 9 9 . 1
  35. Exporting versions as version_info tuples i m p o r

    t p b r . v e r s i o n . . . _ v e r s i o n = p b r . v e r s i o n . V e r s i o n I n f o ( ' M Y P A C K A G E ' ) . s e m a n t i c _ v e r s i o n ( ) _ _ v e r s i o n _ _ = _ v e r s i o n . v e r s i o n _ t u p l e ( ) v e r s i o n = _ v e r s i o n . r e l e a s e _ s t r i n g ( ) . . . > > > M Y P A C K A G E . v e r s i o n ' 0 . 0 . 4 . d e v 1 . g c a 1 8 f 3 9 ' > > > M Y P A C K A G E . _ _ v e r s i o n _ _ ( 0 , 0 , 4 , ' a l p h a ' , 0 )
  36. (planned) Allow disabling pre- release versions for CD folk p

    y t h o n s e t u p . p y s d i s t . . . 1 . 3 . 2 . 0 a 1 p y t h o n s e t u p . p y s d i s t - - n o - r c 1 . 3 . 2 . d e v 3 4 . g $ s h a
  37. Questions? Robert Collins @rbtcollins lifeless on freenode rbtcollins@hp.com