From Perforce To Git: Lessons Learned Lars Schneider Technical Lead GitHub Solutions Autodesk larsxschneider kit3bus [email protected] 

‣ Best known for AutoCAD - 2D and 3D computer-aided design ‣ 33 years in business ‣ 4000 engineers, hundreds of products, terabytes of code and assets ‣ Architecture, engineering and construction (e.g. Revit) ‣ Manufacturing (e.g. Inventor) ‣ Media and entertainment (e.g. Maya) ‣ 3D printing (e.g. Spark) Who is Autodesk? 

Why do we migrate code to Git? ‣ Standardize ‣ Discover ‣ Share/Reuse ‣ Collaborate Quality! 

What you learn? 1. Migrate repositories using git-p4 ‣ History ‣ File encodings ‣ Branches ‣ Binary files 2. Restructure repositories 3. Non-technical issues 

Migration from Perforce (p4) to Git export P4USER="lars" export P4PORT=“tcp:my-perforce-server.com:12345” p4 login mkdir demo && cd demo git-p4 clone //depot@all . --verbose git remote add origin [email protected]:org/demo.git git push origin Demo migration using git-p4: Downloads: https://www.perforce.com https://github.com/git/git/blob/next/git-p4.py 

Perforce client spec ! ! Client: projectX-spec! Owner: lars! Root: /workspaces/temp! Options: noallwrite noclobber nocompress unlocked nomodtime normdir
 SubmitOptions: submitunchanged! LineEnd: local! View:! //depot/projectX/src/... //myspec/src/...
 //depot/tests/projectX/... //myspec/tests/... File:/path/to/client.spec ... p4 login p4 client -i < /path/to/client.spec mkdir demo && cd demo git init git config git-p4.useClientSpec true git config git-p4.client projectX-spec git-p4 clone //depot@all . --verbose ... 

Path case ! 
 Client: myspec! Owner: lars! Root: /workspaces/temp! Options: noallwrite noclobber ...! SubmitOptions: submitunchanged! LineEnd: local! View:! //depot/case/... //myspec/...! File:/path/to/client.spec ! 

Path case ? ! 

Path case ! 

Path case export P4USER="lars"! export P4PORT="tcp:server.com:12345"! ! p4 login! p4 client -i < /path/to/client.spec! ! mkdir demo && cd demo! git init! git config git-p4.useClientSpec true! git config git-p4.client myspec! git config core.ignorecase true" git-p4 clone //depot@all . --verbose! ... ! 

Path case ? ! 

Path case bug ! 

Path case …fixed in Git 2.6 ! ! 

! Path encoding Perforce Git cp1252 (Wester European Windows) UTF-8 (usually on Linux/OSX) UTF-8 UTF-8 

export P4USER="lars"! export P4PORT="tcp:server.com:12345"! ! p4 login! ! mkdir demo && cd demo! git init! git config git-p4.pathEncoding cp1252" git-p4 clone //depot@all . --verbose! git remote add origin [email protected]:org/demo.git! git push origin! ! Windows Path encoding ! 

export P4USER="lars"! export P4PORT="tcp:server.com:12345"! ! p4 login! ! mkdir demo && cd demo! git init! git config git-p4.pathEncoding cp1252" git-p4 clone //depot@all . --verbose! git remote add origin [email protected]:org/demo.git! git push origin! ! Windows Path encoding …in Git “next”, part of 2.7 ?! ! 

UTF16/Unicode file encoding What?!! 

UTF16/Unicode file encoding Perforce text file line ending conversions Windows Unix CR LF LF as defined on migration Git binary file no conversions Windows Unix 

UTF16/Unicode file encoding Git binary file no conversions Windows Unix ! ! Client: myspec! Owner: lars! Root: /workspaces/temp! Options: noallwrite noclobber ...! SubmitOptions: submitunchanged! LineEnd: win! View:! //depot/path/... //myspec/...! File:/path/to/client.spec CR LF 

File encodings ‣ Perforce added UTF16 support in 2007 ‣ Old file revisions might be in a bad state: ! Translation of file content failed near line 1" Details: http://answers.perforce.com/articles/KB/3117 ...! git-p4 clone //depot@all . --verbose" ... …scheduled for Git “next”, part of 2.7 ?! ! 

Branches ‣ Directories (Perforce) vs. Graph (Git) ‣ Git-P4 branch import did not work for me ‣ Poor man’s branch import! 

Branches 1. Import Perforce branch directories as individual Git branches A B Release 1 2 3 Master 

Branches 2. Create a copy of your branch Release 1 2 3 Master A B p4-history/Release 

Branches 3. Squash and rename your branch X Squashed Release 1 2 3 Master A B p4-history/Release 

Branches 4. Create a new Git branch based on Master X Squashed Release 1 2 3 Master A B p4-history/Release Release 

Branches 5. Commit the delta of the squashed branch X 1 2 3 Master A B p4-history/Release Release 

Branches 6. Continue to work with Git branches X 1 2 3 Master A B p4-history/ 4 5 Y Release 

‣ Difficult due to Git’s design ‣ Git extensions to the rescue: ‣ git-annex ‣ git-fat ‣ git-media ‣ Git LFS Git Repo File Server 1011 0101 1101 1011 0101 1101 1011 0101 1101 1011 0101 1101 1011 0101 1101 Pointer Pointer .gitattributes Large binary files Details: https://git-lfs.github.com/ 

Large binary files Yeah!! 

Large binary files export P4USER="lars" export P4PORT="tcp:server.com:12345" p4 login mkdir demo && cd demo git init git config git-p4.largeFileSystem GitLFS git config git-p4.largeFileExtensions mp4 git config git-p4.largeFilePush True git remote add origin [email protected]:org/demo.git git-p4 clone //depot@all . --verbose git push origin ! ! #! # Git LFS (see https://git-lfs.github.com/)! #! *.mp4 filter=lfs -text" File:.gitattributes 

Large binary files What file extensions should you pick? 

Large binary files export P4USER="lars"! export P4PORT="tcp:server.com:12345"! ! p4 login! ! mkdir demo && cd demo! git init! git config git-p4.largeFileSystem GitLFS! git config git-p4.largeFileCompressedThreshold 200k" git config git-p4.largeFilePush True! git remote add origin [email protected]:org/demo.git! git-p4 clone //depot@all . --verbose! git push origin ! ! #! # Git LFS (see https://git-lfs.github.com/)! #! /images/bigscreen.png filter=lfs -text! /libs/gigantic.mp4 filter=lfs -text" File:.gitattributes …scheduled for Git “next”, part of 2.7 ?! 

Restructure repositories Source: https://marc.info/?l=git&m=124121401124923 – Linus Torvalds “Git fundamentally never really looks at less than the whole repo.” 

Exclude undesired files Not part of the project! 

Exclude undesired files ! ! Client: myspec! Owner: lars! Root: /workspaces/temp! Options: noallwrite noclobber nocompress ...! SubmitOptions: submitunchanged! LineEnd: local! View:! //depot/... //myspec/...! -//depot/octocats/... //myspec/octocats/..." File:/path/to/client.spec Source: http://www.perforce.com/perforce/doc.current/manuals/cmdref/views.html 

‣ Speed up Git operations ‣ Reduce time to 
 clone/fetch Exclude undesired files ! ! Client: myspec! Owner: lars! Root: /workspaces/temp! Options: noallwrite noclobber nocompress ...! SubmitOptions: submitunchanged! LineEnd: local! View:! //depot/... //myspec/...! -//depot/octocats/... //myspec/octocats/..." File:/path/to/client.spec Source: http://www.perforce.com/perforce/doc.current/manuals/cmdref/views.html 

Move 3rd party code ‣ Use native package manager (npm, pip, …) ‣ Use Git Submodules ‣ Zip components and upload them to Artifactory 

Extract components ‣ Hard! Usually requires refactoring! ‣ Only viable for independent components which do not share same “pace of development” 

Watchman ‣ Daemon to monitor file system changes, developed by Facebook ‣ Default: Git asks file system for changes ‣ Watchman: Filesystems tells Git what changed ‣ Git proof of concept by David Turner, Twitter Details: https://facebook.github.io/watchman/ https://www.mail-archive.com/git%40vger.kernel.org/msg49593.html 

Biggest technical challenge # of files 

How about non-technical challenges? ‣ new concepts forks staging area remotes hashes rebase Seriously?!! 

How about non-technical challenges? ‣ new concepts ‣ new workflows pull requests feature branches 

How about non-technical challenges? ‣ new concepts ‣ new workflows ‣ new user interface Command line? 80s again?!? SourceTree GitHub 

How about non-technical challenges? ‣ new concepts ‣ new workflows ‣ new user interface adaptive challenges! 

Start a movement ‣ Established a GitHub Experts Team ‣ Created GitHub Q&A page ‣ Hands on training for the first projects ‣ Create a Git fan base! 

Summary: Perforce to Git ‣ Use git-p4, latest version ‣ Use LFS for large files ‣ Try to reduce the # of files ‣ Start a movement 

Thanks Lars Schneider Technical Lead GitHub Solutions Autodesk From Perforce To Git: Lessons Learned larsxschneider kit3bus [email protected]