Robert Collins: Semver and Python with PBR

= = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = =
Robert Collins:
Semver and Python with PBR
= = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = =
@ Kiwi PyCon 2014 - Saturday, 13 Sep 2014 - Track 1
http://kiwi.pycon.org/

**Audience level**

Novice

**Description**

PBR is a setuptools plugin which OpenStack developed to provide simple and consistent minimal-boilerplate build definitions for its projects. Semantic versioning provides simple and robust rules for deciding on version numbers, and PBR will soon be able to manage the entire process of version number assignment in a project... and more.

**Abstract**

PBR is a setuptools plugin which OpenStack developed to provide simple and consistent minimal-boilerplate build definitions for its projects. Now used by all the OpenStack projects, PBR provides integration glue for core features:

- testing
- binary package creation for Linux distributors
- inclusion of files in tarballs
- changelog and authors file creation
- pypi summary creation
- version number creation
- sphinx doc stub creation and manpage enablement
- unified requirements management
- for both easy-install and pip with single-file control

The most interesting part is the version number creation, since coming up with the right version number can be a contentious discussion in some projects. Semver provides simple and robust rules for deciding on version numbers, and I'm in the middle of implementing automation for these in PBR itself, with integration glue to export them in PEP-440, dpkg and rpm format. The only dependencies PBR has are git + a recent pip, so this should be useful for many attendees - and while PBR is an OpenStack invention we're very interested in making sure its useful and reliable for anyone that wants to use it.

**YouTube**

https://www.youtube.com/watch?v=6hjg-Uia5AU

Transcript

1. pbr and semver aka Python Build Reasonableness and Semantic Versioning Robert Collins rbtcollins@hp.com @rbtcollins (Twitter)
2. Setuptools
3. OpenStack Dozens of projects Identical infrastructure code Maintenance nightmare
4. Refactor New project Setuptools plugin Used by all OpenStack projects
5. PBR Consistent setuptools integration glue for all OpenStack projects. Depends on git and a recent pip for installation support.
6. Getting started with PBR
7. setup.py # ! / u s r / b i n / e n v p y t h o n i m p o r t s e t u p t o o l s s e t u p t o o l s . s e t u p ( s e t u p _ r e q u i r e s = [ ' p b r ' ] , p b r = T r u e )
8. setup.cfg [ m e t a d a t a ] n a m e = o s - a p p l y - c o n f i g a u t h o r = O p e n S t a c k a u t h o r - e m a i l = o p e n s t a c k - d e v @ l i s t s . o p e n s t a c k . o r g s u m m a r y = C o n f i g f i l e s f r o m c l o u d m e t a d a t a d e s c r i p t i o n - f i l e = R E A D M E . r s t h o m e - p a g e = h t t p : / / g i t . o p e n s t a c k . o r g / c g i t / o p e n s t a c k / o s - a p p l y - c o n f i g c l a s s i f i e r = D e v e l o p m e n t S t a t u s : : 4 - B e t a . . . . P r o g r a m m i n g L a n g u a g e : : P y t h o n [ f i l e s ] p a c k a g e s = o s _ a p p l y _ c o n f i g [ e n t r y _ p o i n t s ] c o n s o l e _ s c r i p t s = o s - a p p l y - c o n f i g = o s _ a p p l y _ c o n f i g . a p p l y _ c o n f i g : m a i n
9. PBR Features
10. Testing entrypoint integrated p y t h o n s e t u p . p y t e s t
11. git integration Hard requirement but.... automatically picks up all versioned files scans history for changelog and authors file contents
12. PyPI integration Generates a rich summary from README.rst + changelog
13. Documentation Sphinx API glue, and Sphinx manpage output.
14. PIP Pulls requirements from requirements*.txt
15. Versioning Generates pre-release version numbers for you from tags and or config settings
16. Semantic versioning is a simple and clear set of rules for versioning software to clearly indicate compatibility interactions. semver
17. openstack-semver "This is a fork of Semantic Versioning 2.0. The specific changes have to do with the format of pre-release and build labels, specifically to make them not confusing when co-existing with Linux Distribution packaging. Inspiration for the format of the pre-release and build labels came from Python's PEP440."
18. Issues
19. dev builds 1.3.2 tagged 5 local commits 1.3.2.5.g$sha
20. pbr-semver Developers don't know the semver rules we've settled on: tag 1.3.2.a2 should be 1.3.2.0a2
21. CD 1.3.2.0a1 cannot be sorted vs 1.3.2.dev5
22. Binary Packaging Turns out everyone reinvents the same integration and version translation glue... differently.
23. Sort-before 1.3.2.0a1 sorts before 1.3.2 - versions are special. dpkg has ~ rpm has nothing
24. pbr-semver new shiny magic
25. Strict pbr-semver mode Enforce pbr-semver compatible version numbers
26. Epic fail on existing versions
27. Have to parse every version we ever accepted: 1 . 2 . 1 a 1 1 . 2 . 3 . 4 . g 1 2 3 1 . 2 . b 1 1 . 2 b 1 2 . 1 . 0 . r c 1
28. History scanning Scan git history for pseudo headers and calculate new release versions
29. Before: pbr-0.10.1.dev5.gaada1d1
30. L o o k f o r a n d p r o c e s s s e m - v e r p s e u d o h e a d e r s i n g i t A t t h e m o m e n t c a r e f u l c h a n g e l o g r e v i e w i s n e e d e d b y h u m a n d s t o d e t e r m i n e t h e n e x t v e r s i o n n u m b e r . S e m a n t i c v e r s i o n i n g m e a n s t h a t a l l w e n e e d t o k n o w t o g e t a r e a s o n a b l e d e f a u l t n e x t v e r s i o n ( a n d a l o w e r b o u n d o n t h e n e x t v e r s i o n ) i s a r e c o r d o f b u g f i x / d e p r e c a t i o n / f e a t u r e w o r k a n d a p i - b r e a k i n g c o m m i t s . I n t h i s p a t c h w e s t a r t s c a n n i n g f o r s u c h h e a d e r s f r o m t h e g i t h i s t o r y t o r e m o v e t h i s b u r d e n f r o m d e v e l o p e r s / p r o j e c t r e l e a s e m a n a g e r s . H i g h e r v e r s i o n s c a n o f c o u r s e b e u s e d e i t h e r v i a p r e - v e r s i o n i n g o r b y t a g g i n g t h e d e s i r e d v e r s i o n . i m p l e m e n t s : b l u e p r i n t p b r - s e m v e r s e m - v e r : f e a t u r e < - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - C h a n g e - I d : I d 5 e 8 c d 7 2 3 d 5 1 8 6 d 1 b d 8 c 0 1 5 9 9 e a e 8 9 3 3 e 6 f 7 e a 6 d
31. After: pbr-0.11.0.dev5.gaada1d1
32. s e m - v e r : f e a t u r e s e m - v e r : a p i - b r e a k s e m - v e r : d e p r e c a t i o n s e m - v e r : b u g f i x s e m - v e r : b u g f i x , a p i - b r e a k
33. (planned) Doing releases W#rap git tag to make releases trivial p y t h o n s e t u p . p y t a g - r e l e a s e
34. (planned) Doing binary (distro) builds Provide helpers to generate dpkg and RPM versions from a semver version s e t u p . p y d e b - v e r s i o n 1 . 2 . 0 ~ 0 b 1 s e t u p . p y r p m - v e r s i o n 1 . 1 . 9 9 9 9 . 1
35. Exporting versions as version_info tuples i m p o r t p b r . v e r s i o n . . . _ v e r s i o n = p b r . v e r s i o n . V e r s i o n I n f o ( ' M Y P A C K A G E ' ) . s e m a n t i c _ v e r s i o n ( ) _ _ v e r s i o n _ _ = _ v e r s i o n . v e r s i o n _ t u p l e ( ) v e r s i o n = _ v e r s i o n . r e l e a s e _ s t r i n g ( ) . . . > > > M Y P A C K A G E . v e r s i o n ' 0 . 0 . 4 . d e v 1 . g c a 1 8 f 3 9 ' > > > M Y P A C K A G E . _ _ v e r s i o n _ _ ( 0 , 0 , 4 , ' a l p h a ' , 0 )
36. (planned) Allow disabling pre- release versions for CD folk p y t h o n s e t u p . p y s d i s t . . . 1 . 3 . 2 . 0 a 1 p y t h o n s e t u p . p y s d i s t - - n o - r c 1 . 3 . 2 . d e v 3 4 . g $ s h a
37. Questions? Robert Collins @rbtcollins lifeless on freenode rbtcollins@hp.com