Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Gerrit for LibreOffice

Gerrit for LibreOffice

The code review and continuous integration provided by gerrit will help us shape a glorious future for LibreOffice

LibreOffice conference, Berlin, 17th October 2012

Bjoern Michaelsen

October 17, 2012
Tweet

More Decks by Bjoern Michaelsen

Other Decks in Programming

Transcript

  1. 3 Crowdsourcing code review Patches Lost in Space: the old

    workflow A patch is intended to: Be mailed to the dev@-mailinglist Being picked up by a developer with commit rights and be Checked for a valid license Reviewed for Goodness Applied, Build, Tested and Pushed Reported back to the mailing list that this is in now Reality: fdo#40946
  2. 5 Crowdsourcing code review Does it hurt? Duplication of work

    Frustration Scaring away newcomers Makes some kinds of work very hard or painful for everyone
  3. 7 Crowdsourcing code review Tinderboxes are Annoying and Scary Tinderboxes

    are useless when the master has been already broken and still receives commits. Tinderboxes are annoying. Tinderboxes are scary.
  4. 9 Crowdsourcing code review Tinderboxes are Lemmings Tinderboxes have different

    speeds. Tinderboxes know nothing of each other. Tinderboxes cant communicate or coordinate. The slowest and rarest tinderboxes (Windows) are the most valueable.
  5. 11 Crowdsourcing code review How does Gerrit help? Possibilities with

    Gerrit: Automatic Patch Uploads Intergration in a wide range of communication channels Human Review Tinderbox Review Testsuite Review Thus: Find errors early and before they cause too much pain.
  6. 13 Crowdsourcing code review Reviews Code Review: +2 → looks

    good to me -2 → violent opposition Later: +1 → at least doesnt look like a trojan to me Verified: (blackbox testing: observations without looking at the code) +1 → passes all tests I subjected it to recommended: at least building on one platform -1 → failed in some way (building, testing)
  7. 14 Crowdsourcing code review Gerrit Maildrop Send to [email protected] Subject:

    [PATCH] for branch master Subject: [CHERRY-PICK] commit 12345678abcdef for branch libreoffice-3-6
  8. 15 Crowdsourcing code review Better tools might mean less interaction

    Yes, better tools sometimes result in less interaction The “saved” interactions are usually of the annoying kind anyway: Repetitive, Administrative and Blocking Lets keep up a good amount of personal interaction, with the time saved from the better tools: Public personal comments
  9. 16 Crowdsourcing code review Gerrit: Bringing it all together Gerrit

    is now integrated with: Mailing lists Bugzilla (oneway) IRC git commandline (git review and logerrit) Still needs to be integrated with: (volunteers welcome!) RSS Twitter (trivial with twitterfeed once RSS is there) Identica g+
  10. 19 Gerrit for Libreoffice Infrastructure Infrastructure behind gerrit (1/2) LO-Server

    runs on Ubuntu 11.10 Will be migrated to 12.04 soon :-) Services are separated on Linux LXC-Containers Every important Service has its own LXC-Container Details on IS see IS-Talk Friday (F. Effenberger / A.Werner)
  11. 20 Gerrit for Libreoffice Infrastructure Infrastructure behind gerrit (2/2) Version

    of gerrit: 2.4.2 Will be upgraded as soon as 2.5 is available Database: PostgreSQL 9.1 Apache 2.2 as Frontend-Server Additional self-written Plugins
  12. 21 Gerrit for Libreoffice Infrastructure Installation of gerrit Distribution of

    the LXC-Container (Puppet) Installtion of Dependencies (Apache / PostgreSQL) Download of gerrit from google-Sites Installation of gerrit based on http://gerrit-documentation.googlecode.com/svn/Documentation/2.4.2/install.html Future: our own Puppet-Plugin for Installation / Upgrade
  13. 22 Gerrit for Libreoffice Infrastructure Backup TDF has a simple

    Backup-Structure based on BackupPC Backup is done twice a day Backup is stored on two Backup-Machines Future (after Migration to 12.04): LVM-Snapshot-Backup of LXC-Container + Snapshot of Database Downtime per Backup about 1 sec.
  14. 23 Gerrit for Libreoffice Infrastructure Disaster-Recovery Copy the whole LXC-Container

    from Backup-Machine to LXC-Host incl the whole Database Let it run :-) Takes about 10 – 15 Min, depends on size of git-Repository
  15. 25 LibreOffice Berlin 2012 Conference Presentation Template Decentralized CI Infrastructure

    (without gerrit) 1/3 Independent tinderboxes that periodically check master and release branches Multiple checks for the same commits on the same platforms possible Build and configuration logic is located locally in tinbuild2 script Report results to tinderbox master per fire and forget Tinderbox master presents results in a web page
  16. 26 LibreOffice Berlin 2012 Conference Presentation Template Decentralized CI Infrastructure

    (without gerrit) 2/3 Tinderbox_1 Tinderbox_n Tinderbox_2 Tinderbox_... Repository Tinderbox Master Web page
  17. 28 LibreOffice Berlin 2012 Conference Presentation Template Decentralized CI Infrastructure

    with gerrit 1/3 Some kind of queue manager is needed to distribute checks jobs between tinderboxes One patch set must be checked only once for each platform and configuration Tinbuild2 must be teached to talk to queue manager Results from all tinderboxes are gathered and combined result is reported back to gerrit A patch set is approved only when all check jobs were successful.
  18. 29 LibreOffice Berlin 2012 Conference Presentation Template Decentralized CI Infrastructure

    with gerrit 2/3 Tinderbox_1 Tinderbox_n Tinderbox_2 Tinderbox_... Queue manager gerrit
  19. 30 LibreOffice Berlin 2012 Conference Presentation Template Decentralized CI Infrastructure

    with gerrit 3/3 Tinderbox_1 Tinderbox_n Tinderbox_2 Tinderbox_... Repository Tinderbox Master Web page Queue manager gerrit Reusing existing infrastructure Tinbuild2 teached to talk to queue manager Different strategies for gerrit patch verfication vs. master check Prefer gerrit verification Prefer master check
  20. 31 LibreOffice Berlin 2012 Conference Presentation Template Queue manager's feature

    set Consumption gerrit-stream events Communication API/protocol for tinderboxen Job Scheduler Build log publication and visualization Internal pipeline visualization Documentation Nice to have: Gerrit web ui extension check job results are reported in own table on change page in gerrit during review process specific configuration can be selected to be checked, i. e. with-java if java stack was changed in change context
  21. 32 LibreOffice Berlin 2012 Conference Presentation Template Gerrit Plugin Architecture

    Since upcoming gerrit 2.5 plugin architecture is introduced What can a gerrit plugin extend? New ssh commands under plugin's own namespace New plugin web pages under plugin's own namespace Extend database Highly experimental and not in upcoming release 2.5: web ui extension of core pages (GWT and javascript plugin types) Dog fooding: Replication is a gerrit plugin now and not a core feature https://review.gerrit.net/plugins/plugin-name-1.0/page.html ssh gerrit plugin-name-1.0 some-really-cool-command --foo --bar baz
  22. 33 LibreOffice Berlin 2012 Conference Presentation Template Gerrit-buildbot: queue manager

    as gerrit plugin 1/6 Tinderbox_1 Tinderbox_n Tinderbox_2 Tinderbox_... Gerrit buildbot-plugin
  23. 34 LibreOffice Berlin 2012 Conference Presentation Template Gerrit-buildbot: queue manager

    as gerrit plugin 2/6 Consumption gerrit stream events Trivially as gerrit plugin: just a callback Communication API/protocol (for tinbuild2 and other clients) Gerrit ssh command: Job Scheduler Internally job queue is maintained and can be polled per ssh command: Result reporting and build log publication per ssh: ssh logerrit buildbot-1.0 get-task --project core --platform linux ssh logerrit buildbot-1.0 show-queue –project core cat log | ssh logerrit buildbot-1.0 report --ticket 4711 --failed
  24. 35 LibreOffice Berlin 2012 Conference Presentation Template Gerrit-buildbot: queue manager

    as gerrit plugin 3/6 Build log visualization: web page extension
  25. 36 LibreOffice Berlin 2012 Conference Presentation Template Gerrit-buildbot: queue manager

    as gerrit plugin 4/6 Documentation: per ssh ssh logerrit buildbot-1.0 report --help buildbot-1.0 report [--] [--failed (-f)] [--help (-h)] [--log (-l) -|LOG] [--succeed (-s)] --ticket (-t) TICKET -- : end of options --failed (-f) : specify this option if job failed --help (-h) : display this help text --log (-l) -|LOG : url of the job log page or - for standard input --succeed (-s) : specify this option if job was successfull --ticket (-t) TICKET : ticket of the job
  26. 38 LibreOffice Berlin 2012 Conference Presentation Template Gerrit-buildbot: queue manager

    as gerrit plugin 6/6 Result are presented in gerrit's comment table Nice to have: Implement CI verification status table on gerrit's change page (once web ui plugin can extend core pages) Don't mix human's review and CI's verification outcome
  27. 39 Gerrit for Libreoffice Infrastructure All text and image content

    in this document is licensed under the Creative Commons Attribution-Share Alike 3.0 License (unless otherwise specified). "LibreOffice" and "The Document Foundation" are registered trademarks. Their respective logos and icons are subject to international copyright laws. The use of these therefore is subject to the trademark policy. ??? Questions ???