Share
Embed
From Perforce To Git: Lessons Learned
GitHub Universe Build Breakout Session. Find the recording here:
http://www.ustream.tv/recorded/74637923
Lars Schneider
October 02, 2015
Transcript
47 slides
-
From Perforce To Git: Lessons Learned Lars Schneider Technical Lead
GitHub Solutions Autodesk larsxschneider kit3bus [email protected] -
‣ Best known for AutoCAD - 2D and 3D computer-aided
design ‣ 33 years in business ‣ 4000 engineers, hundreds of products, terabytes of code and assets ‣ Architecture, engineering and construction (e.g. Revit) ‣ Manufacturing (e.g. Inventor) ‣ Media and entertainment (e.g. Maya) ‣ 3D printing (e.g. Spark) Who is Autodesk? -
Why do we migrate code to Git? ‣ Standardize ‣
Discover ‣ Share/Reuse ‣ Collaborate Quality! -
What you learn? 1. Migrate repositories using git-p4 ‣ History
‣ File encodings ‣ Branches ‣ Binary files 2. Restructure repositories 3. Non-technical issues -
Migration from Perforce (p4) to Git export P4USER="lars" export P4PORT=“tcp:my-perforce-server.com:12345”
p4 login mkdir demo && cd demo git-p4 clone //depot@all . --verbose git remote add origin [email protected]:org/demo.git git push origin Demo migration using git-p4: Downloads: https://www.perforce.com https://github.com/git/git/blob/next/git-p4.py -
Perforce client spec ! ! Client: projectX-spec! Owner: lars! Root:
/workspaces/temp! Options: noallwrite noclobber nocompress unlocked nomodtime normdir SubmitOptions: submitunchanged! LineEnd: local! View:! //depot/projectX/src/... //myspec/src/... //depot/tests/projectX/... //myspec/tests/... File:/path/to/client.spec ... p4 login p4 client -i < /path/to/client.spec mkdir demo && cd demo git init git config git-p4.useClientSpec true git config git-p4.client projectX-spec git-p4 clone //depot@all . --verbose ... -
Path case ! Client: myspec! Owner: lars! Root: /workspaces/temp!
Options: noallwrite noclobber ...! SubmitOptions: submitunchanged! LineEnd: local! View:! //depot/case/... //myspec/...! File:/path/to/client.spec ! -
Path case export P4USER="lars"! export P4PORT="tcp:server.com:12345"! ! p4 login! p4
client -i < /path/to/client.spec! ! mkdir demo && cd demo! git init! git config git-p4.useClientSpec true! git config git-p4.client myspec! git config core.ignorecase true" git-p4 clone //depot@all . --verbose! ... ! -
! Path encoding Perforce Git cp1252 (Wester European Windows) UTF-8
(usually on Linux/OSX) UTF-8 UTF-8 -
export P4USER="lars"! export P4PORT="tcp:server.com:12345"! ! p4 login! ! mkdir demo
&& cd demo! git init! git config git-p4.pathEncoding cp1252" git-p4 clone //depot@all . --verbose! git remote add origin [email protected]:org/demo.git! git push origin! ! Windows Path encoding ! -
export P4USER="lars"! export P4PORT="tcp:server.com:12345"! ! p4 login! ! mkdir demo
&& cd demo! git init! git config git-p4.pathEncoding cp1252" git-p4 clone //depot@all . --verbose! git remote add origin [email protected]:org/demo.git! git push origin! ! Windows Path encoding …in Git “next”, part of 2.7 ?! ! -
UTF16/Unicode file encoding Perforce text file line ending conversions Windows
Unix CR LF LF as defined on migration Git binary file no conversions Windows Unix -
UTF16/Unicode file encoding Git binary file no conversions Windows Unix
! ! Client: myspec! Owner: lars! Root: /workspaces/temp! Options: noallwrite noclobber ...! SubmitOptions: submitunchanged! LineEnd: win! View:! //depot/path/... //myspec/...! File:/path/to/client.spec CR LF -
File encodings ‣ Perforce added UTF16 support in 2007 ‣
Old file revisions might be in a bad state: ! Translation of file content failed near line 1" Details: http://answers.perforce.com/articles/KB/3117 ...! git-p4 clone //depot@all . --verbose" ... …scheduled for Git “next”, part of 2.7 ?! ! -
Branches ‣ Directories (Perforce) vs. Graph (Git) ‣ Git-P4 branch
import did not work for me ‣ Poor man’s branch import! -
Branches 1. Import Perforce branch directories as individual Git branches
A B Release 1 2 3 Master -
Branches 2. Create a copy of your branch Release 1
2 3 Master A B p4-history/Release -
Branches 3. Squash and rename your branch X Squashed Release
1 2 3 Master A B p4-history/Release -
Branches 4. Create a new Git branch based on Master
X Squashed Release 1 2 3 Master A B p4-history/Release Release -
Branches 5. Commit the delta of the squashed branch X
1 2 3 Master A B p4-history/Release Release -
Branches 6. Continue to work with Git branches X 1
2 3 Master A B p4-history/ 4 5 Y Release -
‣ Difficult due to Git’s design ‣ Git extensions to
the rescue: ‣ git-annex ‣ git-fat ‣ git-media ‣ Git LFS Git Repo File Server 1011 0101 1101 1011 0101 1101 1011 0101 1101 1011 0101 1101 1011 0101 1101 Pointer Pointer .gitattributes Large binary files Details: https://git-lfs.github.com/ -
Large binary files export P4USER="lars" export P4PORT="tcp:server.com:12345" p4 login mkdir
demo && cd demo git init git config git-p4.largeFileSystem GitLFS git config git-p4.largeFileExtensions mp4 git config git-p4.largeFilePush True git remote add origin [email protected]:org/demo.git git-p4 clone //depot@all . --verbose git push origin ! ! #! # Git LFS (see https://git-lfs.github.com/)! #! *.mp4 filter=lfs -text" File:.gitattributes -
Large binary files export P4USER="lars"! export P4PORT="tcp:server.com:12345"! ! p4 login!
! mkdir demo && cd demo! git init! git config git-p4.largeFileSystem GitLFS! git config git-p4.largeFileCompressedThreshold 200k" git config git-p4.largeFilePush True! git remote add origin [email protected]:org/demo.git! git-p4 clone //depot@all . --verbose! git push origin ! ! #! # Git LFS (see https://git-lfs.github.com/)! #! /images/bigscreen.png filter=lfs -text! /libs/gigantic.mp4 filter=lfs -text" File:.gitattributes …scheduled for Git “next”, part of 2.7 ?! -
Restructure repositories Source: https://marc.info/?l=git&m=124121401124923 – Linus Torvalds “Git fundamentally never
really looks at less than the whole repo.” -
Exclude undesired files ! ! Client: myspec! Owner: lars! Root:
/workspaces/temp! Options: noallwrite noclobber nocompress ...! SubmitOptions: submitunchanged! LineEnd: local! View:! //depot/... //myspec/...! -//depot/octocats/... //myspec/octocats/..." File:/path/to/client.spec Source: http://www.perforce.com/perforce/doc.current/manuals/cmdref/views.html -
‣ Speed up Git operations ‣ Reduce time to
clone/fetch Exclude undesired files ! ! Client: myspec! Owner: lars! Root: /workspaces/temp! Options: noallwrite noclobber nocompress ...! SubmitOptions: submitunchanged! LineEnd: local! View:! //depot/... //myspec/...! -//depot/octocats/... //myspec/octocats/..." File:/path/to/client.spec Source: http://www.perforce.com/perforce/doc.current/manuals/cmdref/views.html -
Move 3rd party code ‣ Use native package manager (npm,
pip, …) ‣ Use Git Submodules ‣ Zip components and upload them to Artifactory -
Extract components ‣ Hard! Usually requires refactoring! ‣ Only viable
for independent components which do not share same “pace of development” -
Watchman ‣ Daemon to monitor file system changes, developed by
Facebook ‣ Default: Git asks file system for changes ‣ Watchman: Filesystems tells Git what changed ‣ Git proof of concept by David Turner, Twitter Details: https://facebook.github.io/watchman/ https://www.mail-archive.com/git%40vger.kernel.org/msg49593.html -
How about non-technical challenges? ‣ new concepts forks staging area
remotes hashes rebase Seriously?!! -
How about non-technical challenges? ‣ new concepts ‣ new workflows
pull requests feature branches -
How about non-technical challenges? ‣ new concepts ‣ new workflows
‣ new user interface Command line? 80s again?!? SourceTree GitHub -
How about non-technical challenges? ‣ new concepts ‣ new workflows
‣ new user interface adaptive challenges! -
Start a movement ‣ Established a GitHub Experts Team ‣
Created GitHub Q&A page ‣ Hands on training for the first projects ‣ Create a Git fan base! -
Summary: Perforce to Git ‣ Use git-p4, latest version ‣
Use LFS for large files ‣ Try to reduce the # of files ‣ Start a movement -
Thanks Lars Schneider Technical Lead GitHub Solutions Autodesk From Perforce
To Git: Lessons Learned larsxschneider kit3bus [email protected]