Tracking upstream with Gerrit

An approach to syncing with upstream projects, that is compatible with a team using Gerrit/Jenkins to review, and re-check changes being carried.

Transcript

1. Tracking Upstream using Gerrit Darragh Bailey, daragh.bailey@gmail.com January 16, 2014 daragh.bailey@gmail.com Tracking Upstream January 16, 2014 1 / 20
2. What’s the Story? The Problem: Basing a product on a external project Hit a bug that needs to be fixed So? Ideal is fix upstream and consume Sometimes the fix is incorrect and risky for upstream Moving forward introduces risk, including changes you haven’t verified Reality: Frequently need to be able to carry local changes daragh.bailey@gmail.com Tracking Upstream January 16, 2014 2 / 20
3. The CI Process in Use daragh.bailey@gmail.com Tracking Upstream January 16, 2014 3 / 20
4. What do we really want? Avoid reviewing diffs of diffs (review patches + series file) Allow developers to develop changes without needing to prep the project after clone Be able to forward port changes without requiring devs to perform complex rebase of local branches (no ’rebase –onto’) Be able to re-review all changes Be able to re-test each change Allow porting to be resumed by different developers Automate dropping of local changes that were upstreamed Work with desired toolchain daragh.bailey@gmail.com Tracking Upstream January 16, 2014 4 / 20
5. Options? Use patch series Merge upstream into current Use rebase? daragh.bailey@gmail.com Tracking Upstream January 16, 2014 5 / 20
6. What tools are in there in this space? stgit - http://www.procode.org/stgit/ topgit - https://github.com/greenrd/topgit guilt - http://repo.or.cz/w/guilt.git quilt - http://savannah.nongnu.org/projects/quilt git-buildpackage - https://sigxcpu.org/piki/projects/git-buildpackage/ git-dpm - http://git-dpm.alioth.debian.org/ None really worked well with Gerrit! daragh.bailey@gmail.com Tracking Upstream January 16, 2014 6 / 20
7. Merge upstream into current Can mean large conflicts Needs an expert maintainer! Changes can become split between original commit and code resolved in the merge conflict http://stackoverflow.com/questions/1461909/evil-merges-in-git Merges can hide regressions Both parents work correctly But one side reworks code that the other expects behaves differently Result breaks at merge but it’s non-trivial to determine where in the history of either side the problem was introduced Discarded as too likely to go wrong and also doesn’t help encouraging upstreaming of changes! daragh.bailey@gmail.com Tracking Upstream January 16, 2014 7 / 20
8. Could rebase work? Git already can replay local changes Some simple tests show that it works Encourages tracking of local changes More likely to contribute upstream Would support re-reviewing of changes But Problematic: Results in many hanging branches, or Rebasing already published branch used by team locally -> DANGEROUS daragh.bailey@gmail.com Tracking Upstream January 16, 2014 8 / 20
9. Could ’git merge’ on top of ’git rebase’ help? There is an ’ours’ strategy for git merge If we could merge and replace our mainline with upstream + carried patches Then any git pull would have sufficient information for developer changes to be rebased without a complex rebase daragh.bailey@gmail.com Tracking Upstream January 16, 2014 9 / 20
10. The rebase daragh.bailey@gmail.com Tracking Upstream January 16, 2014 10 / 20
11. The merge daragh.bailey@gmail.com Tracking Upstream January 16, 2014 11 / 20
12. Slight Drawback This results in a slightly confusing history - first parent is not the original mainline Since git has no ’theirs’ merge strategy, we need to invert the merge git merge -s ours <import-branch> git diff --no-color --binary HEAD^2 | \ git apply -R --index --whitespace=nowarn - git commit --amend -C HEAD daragh.bailey@gmail.com Tracking Upstream January 16, 2014 12 / 20
13. The Result daragh.bailey@gmail.com Tracking Upstream January 16, 2014 13 / 20
14. Why not... Simply replace master straight away with upstream and then rebase the changes to be carried? daragh.bailey@gmail.com Tracking Upstream January 16, 2014 14 / 20
15. Doesn’t work well with Gerrit Gerrit uses a ’ChangeId’ inserted into the end of each commit message The combination of project, branch and ChangeId must be unique Regresses mainline first daragh.bailey@gmail.com Tracking Upstream January 16, 2014 15 / 20
16. Playing nice with Gerrit import/<git-describe> daragh.bailey@gmail.com Tracking Upstream January 16, 2014 16 / 20
17. The Implementation Written in Python Uses generators to apply multiple filters to each commit Supports including multiple separate packaging branches when importing from upstream (spec file?) Is not Open Source (Sorry) daragh.bailey@gmail.com Tracking Upstream January 16, 2014 17 / 20
18. Demo? daragh.bailey@gmail.com Tracking Upstream January 16, 2014 18 / 20
19. Final Thoughts Worth looking at git-buildpackage and git-dpm to see if they can support this approach in the future This isn’t necessary for every project, but it did help solve our problems I hope that it will be contributed back to the community at some stage daragh.bailey@gmail.com Tracking Upstream January 16, 2014 19 / 20
20. Questions? daragh.bailey@gmail.com Tracking Upstream January 16, 2014 20 / 20