$30 off During Our Annual Pro Sale. View Details »

The quantitative state of OPNFV

The quantitative state of OPNFV

Presentation at the OPNFV Design Summit in Berlin, on June 20th 2016.

More Decks by Jesus M. Gonzalez-Barahona

Other Decks in Technology

Transcript

  1. The Quantitative State of OPNFV
    Jesus M. Gonzalez-Barahona
    [email protected] @jgbarah
    Bitergia / URJC
    This slidedeck: http://speakerdeck.com/jgbarah
    OPNFV Design Summit
    Berlin (Germany), June 20th 2016
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of OPNFV June 2016 1 / 53

    View Slide

  2. Open
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of OPNFV June 2016 2 / 53

    View Slide

  3. Software development
    http://xkcd.com/844/
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of OPNFV June 2016 3 / 53

    View Slide

  4. Analytics
    https://en.wikipedia.org/wiki/Charles_Joseph_Minard
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of OPNFV June 2016 4 / 53

    View Slide

  5. OPNFV
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of OPNFV June 2016 5 / 53

    View Slide

  6. Open Development Analytics for OPNFV
    Slidedeck: http://speakerdeck.com/jgbarah
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of OPNFV June 2016 6 / 53

    View Slide

  7. Structure of the presentation
    1 A bit of context
    2 Data, data, data, and some analysis
    3 Activity: what are you doing?
    4 Performance is key!
    5 Diversity (persons, organizations)
    6 Comparisons are odious
    7 Bonus track
    8 Summary
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of OPNFV June 2016 7 / 53

    View Slide

  8. A bit of context
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of OPNFV June 2016 8 / 53

    View Slide

  9. Me and my two hats
    Uni Rey Juan Carlos:
    LibreSoft research team
    Understanding free, open source software
    Data analytics approach
    Bitergia:
    From research to the real world
    Understanding software development
    Data analytics approach
    http://gsyc.es/~jgb
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of OPNFV June 2016 9 / 53

    View Slide

  10. The company
    The software development analytics company
    dashboards
    reports
    consultancy
    ...
    http://bitergia.com
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of OPNFV June 2016 10 / 53

    View Slide

  11. Data, data, data, and some
    analysis
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of OPNFV June 2016 11 / 53

    View Slide

  12. Open development analytics for OPNFV
    All the data presented
    in this talk
    is available online!!!
    And much more...
    Note: data does for late June and Q2 is still missing
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of OPNFV June 2016 12 / 53

    View Slide

  13. The OPNFV dashboard
    http://projects.bitergia.com/opnfv
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of OPNFV June 2016 13 / 53

    View Slide

  14. The OPNFV reports
    https://wiki.opnfv.org/display/DEV/Quarterly+Reports
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of OPNFV June 2016 14 / 53

    View Slide

  15. The Bitergia Analytics dashboard (WIP)
    https://dashboard.bitergia.com/opnfv
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of OPNFV June 2016 15 / 53

    View Slide

  16. Transparency analysis
    Data sources available:
    Source code management (git)
    Code review (Gerrit)
    Issue tracking (Jira)
    Mailing lists
    Instant messaging (IRC)
    Questions & answers (Askbot)
    Automated testing (Jenkins)
    Visualizations, data, and analytics
    available from the project
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of OPNFV June 2016 16 / 53

    View Slide

  17. Activity: what are you doing?
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of OPNFV June 2016 17 / 53

    View Slide

  18. Git: commits over time
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of OPNFV June 2016 18 / 53

    View Slide

  19. Git: active authors over time
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of OPNFV June 2016 19 / 53

    View Slide

  20. Git: active authors over time (by participation)
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of OPNFV June 2016 20 / 53

    View Slide

  21. Code review (top: changesets, bottom: submitters)
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of OPNFV June 2016 21 / 53

    View Slide

  22. Tickets (closed / opened)
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of OPNFV June 2016 22 / 53

    View Slide

  23. Mailing lists (top: messages, bottom: senders)
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of OPNFV June 2016 23 / 53

    View Slide

  24. IRC (top: messages, bottom: identities)
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of OPNFV June 2016 24 / 53

    View Slide

  25. Performance is key!
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of OPNFV June 2016 25 / 53

    View Slide

  26. Review: Changesets per status
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of OPNFV June 2016 26 / 53

    View Slide

  27. Review: pending changesets / actors
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of OPNFV June 2016 27 / 53

    View Slide

  28. Review: Patchsets per changeset
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of OPNFV June 2016 28 / 53

    View Slide

  29. Review: Time to merge, days
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of OPNFV June 2016 29 / 53

    View Slide

  30. Review: Time open, days (top: 50%, bottom: 85%)
    Time open for May 2016: 0.21 (50%), 5.04 (85%)
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of OPNFV June 2016 30 / 53

    View Slide

  31. Review: Time waiting for reviewer, days
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of OPNFV June 2016 31 / 53

    View Slide

  32. Review: Time waiting for submitter, days
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of OPNFV June 2016 32 / 53

    View Slide

  33. Review: current active backlog by month, week
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of OPNFV June 2016 33 / 53

    View Slide

  34. Review: changesets (efficiency)
    Efficiency: fraction of changesets abandoned or merged
    with respect to those opened
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of OPNFV June 2016 34 / 53

    View Slide

  35. Tickets (efficiency)
    Efficiency: fraction of tickets closed
    with respect to those opened
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of OPNFV June 2016 35 / 53

    View Slide

  36. Diversity (persons,
    organizations)
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of OPNFV June 2016 36 / 53

    View Slide

  37. Developers (pony factor)
    15 developers, 50% commits
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of OPNFV June 2016 37 / 53

    View Slide

  38. Organizations (elephant factor)
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of OPNFV June 2016 38 / 53

    View Slide

  39. Organizations (elephant factor)
    3 organizations, 50% commits
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of OPNFV June 2016 39 / 53

    View Slide

  40. Comparisons are odious
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of OPNFV June 2016 40 / 53

    View Slide

  41. OpenStack vs OPNFV (first years)
    OpenStack
    Launched by
    Rackspace, NASA on
    July 19th, 2010
    21 months:
    2010-07 – 2012-03
    http://www.openstack.org/blog/
    2010/07/introducing-openstack/
    OPNFV
    Launched by
    Linux Foundation on
    September 30th, 2014
    21 months:
    2014-10 – 2016-06
    https://www.opnfv.org/news-faq/
    press-release/2014/09/
    telecom-industry-and-vendors-unite-bu
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of OPNFV June 2016 41 / 53

    View Slide

  42. Some parameters, 21th month
    OPNFV OpenStack
    Commits 8,814 23,293
    Commits/mo(a) 700 1,500
    Commits/mo (12mo)(b) 250 → 700 (x2.8) 1,200 → 1,500 (x1.25)
    Authors 203 452
    Authors/mo(a) 80 150
    Pony factor(c) 15 26
    Elephant factor(d) 3 1
    (a) Approximate
    (b) Approximate three-month average year over year
    (c) Pony factor: min. number authors with 50% commits
    (d) Elephant factor: min. number organizations with 50% commits
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of OPNFV June 2016 42 / 53

    View Slide

  43. Commits, 21th month (top: OPNFV, bottom: OpenStack)
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of OPNFV June 2016 43 / 53

    View Slide

  44. Authors, 21th month (top: OPNFV, bottom: OpenStack)
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of OPNFV June 2016 44 / 53

    View Slide

  45. Organizations, 21th month
    (left: OPNFV, : right: OpenStack)
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of OPNFV June 2016 45 / 53

    View Slide

  46. Some parameters, processes
    OPNFV OpenStack
    (2016-Q2) (2015-Q4)
    Time-to-merge (median, days) 0.30 2.38
    Time-to-merge (mean, days) 3.40 11.83
    Patchsets per changeset (median, days) 1.0 2.0
    Patchsets per changeset (mean, days) 1.79 3.86
    Efficiency (changesets) 0.94 0.78
    Efficiency (tickets) 0.57 0.44
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of OPNFV June 2016 46 / 53

    View Slide

  47. Bonus track
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of OPNFV June 2016 47 / 53

    View Slide

  48. Testing: Jenkins data (preview)
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of OPNFV June 2016 48 / 53

    View Slide

  49. Summary
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of OPNFV June 2016 49 / 53

    View Slide

  50. Summarizing...
    OPNFV provides open
    development analytics:
    You don’t need others to provide
    conclusions
    You can do that yourself!
    Slidedeck: http://speakerdeck.com/jgbarah
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of OPNFV June 2016 50 / 53

    View Slide

  51. License
    c 2016 Bitergia
    Some rights reserved.
    This presentation is distributed under the
    “Attribution-ShareAlike 3.0” license, by Creative Commons,
    available at
    http://creativecommons.org/licenses/by-sa/3.0/
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of OPNFV June 2016 51 / 53

    View Slide

  52. Credits (1)
    “Man With Two Hats”
    Statue by Henk Visch, located in Otawa, Canada
    Picture by Lezumbalaberenjena in Wikimedia Commons
    License: Public domain
    https://commons.wikimedia.org/wiki/File:
    Man_With_Two_Hats_Ottawa_Statue_by_lezumbalaberenjena.jpg
    “Napoleon’s Russian campaign of 1812”
    Original by Charles Minard
    License: Public domain
    https://en.wikipedia.org/wiki/Charles_Joseph_Minard#/media/File:
    Minard.png
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of OPNFV June 2016 52 / 53

    View Slide

  53. Credits (2)
    “Aged Come In We’re Open”
    Picture by Czarina Alegre in Flickr
    License: Creative Commons Attribution 2.0
    https://flic.kr/p/fjGamh
    “Good code”
    Comic by Randall Munroe, XKCD 844
    License: Creative Commons Attribution-NonCommercial 2.5
    http://xkcd.com/844/
    Charts and tables are derived from Bitergia dashboards and reports for
    OPNFV
    Jesus Gonzalez-Barahona (Bitergia) The Quantitative State of OPNFV June 2016 53 / 53

    View Slide