Tweet
Tweet

Slide 1

Slide 1 text

GIT LFS - How to handle large Git repos? - Lars Schneider Technical Lead 
 for Git at Autodesk Git and Git LFS contributor @kit3bus

Slide 2

Slide 2 text

What is a large Git repo?

Slide 3

Slide 3 text

Large number of files https://github.com/Microsoft/GVFS

Slide 4

Slide 4 text

Large number of commits

Slide 5

Slide 5 text

Large files

Slide 6

Slide 6 text

Why are large files a problem? = 100 MB video file

Slide 7

Slide 7 text

Clone: Large files in Git master 301MB + 100 MB + 100 MB 1 MB 100 MB 1 MB 100 MB 1 MB 100 MB

Slide 8

Slide 8 text

301 MB 
 x 
 Number of Engineers Clone: Large files in Git 1 10 1 10 1 10 ... Git Server

Slide 9

Slide 9 text

Large files in Git LFS Large File Storage
 https://git-lfs.github.com

Slide 10

Slide 10 text

LFS Server Clone: Large files in Git LFS 1MB master 1 MB 100 MB 1 MB 1 MB 100 MB 0 MB 0 MB 0 MB 100 MB

Slide 11

Slide 11 text

LFS Server Large files in Git LFS master 1 MB 100 MB 1 MB 1 MB 100 MB 0 MB 0 MB 0 MB 100 MB Clone: 1 MB + Checkout: 100 MB

Slide 12

Slide 12 text

LFS Server Large files in Git LFS master 1 MB 100 MB 1 MB 1 MB 100 MB 0 MB 0 MB 0 MB 100 MB Clone: 1 MB + Checkout: 100 MB 60% saving!

Slide 13

Slide 13 text

How to use Git LFS?

Slide 14

Slide 14 text

How to use Git LFS? tracking only based on file name, file size is not relevant! |- .git | |- code.cpp - readme.md - video.mp4 $ git lfs track "*.mp4"

Slide 15

Slide 15 text

How to use Git LFS? |- .git |- .gitattributes |- code.cpp - readme.md - video.mp4 $ git lfs track "*.mp4" $ git add .

Slide 16

Slide 16 text

How to use Git LFS? |- .git |- .gitattributes |- code.cpp - readme.md - video.mp4 $ git lfs track "*.mp4" $ git add . $ git commit

Slide 17

Slide 17 text

How to use Git LFS? |- .git |- .gitattributes |- code.cpp - readme.md - video.mp4 $ git lfs track "*.mp4" $ git add . $ git commit $ git push

Slide 18

Slide 18 text

How to use Git LFS? |- .git |- .gitattributes |- code.cpp - readme.md - video.mp4 $ git lfs track "*.mp4" $ git add . $ git commit $ git push ! That's it!

Slide 19

Slide 19 text

Autodesk uses Git LFS

Slide 20

Slide 20 text

Autodesk uses 
 Git LFS • Best known for AutoCAD
 2D and 3D computer-aided design
 • 35 years in business
 • 4000 engineers, hundreds of products, terabytes of code and asset data

Slide 21

Slide 21 text

• Integration Test Data
 (3D Models, ...)
 • Auxiliary Data
 (Documentation, Images, Videos, ...)
 • Build Artifacts
 (not recommended) Autodesk uses 
 Git LFS

Slide 22

Slide 22 text

What have we learned ? 22

Slide 23

Slide 23 text

Developer (INCLUDES DESIGNER, TESTER, ...)

Slide 24

Slide 24 text

What is too large for Git?

Slide 25

Slide 25 text

What files should go to LFS? Mon Tue Wed Files that 
 do not compress well and change frequently Don't worry about files smaller than 500kb

Slide 26

Slide 26 text

git lfs track "*.png" How to track Git LFS files? Use Git 2.11+ and Git LFS 1.5+ for 80x speed improvement! 
 http://autode.sk/git-filter

Slide 27

Slide 27 text

git lfs track "*.lfs.*" e.g. /images/mountain.lfs.png How to track Git LFS files?

Slide 28

Slide 28 text

git lfs track "/big/*" e.g. /big/mountain.png How to track Git LFS files?

Slide 29

Slide 29 text

git lfs track "/mountain.png" How to track Git LFS files?

Slide 30

Slide 30 text

How to track Git LFS files? git lfs track "*.png" Case sensitive: Win/macOS: demo.PNG Linux: demo.PNG

Slide 31

Slide 31 text

How to track Git LFS files? git lfs track "*.png" Case sensitive: git lfs track "*.[pP][nN][gG]" Case in-sensitive: Use "git lfs ls-files" to check what files are tracked! Win/macOS: demo.PNG Linux: demo.PNG

Slide 32

Slide 32 text

Git LFS Gotchas

Slide 33

Slide 33 text

Gotchas Speed up Clones git lfs clone I'm working on making 
 this wrapper obsolete!

Slide 34

Slide 34 text

Gotchas Setup your Git credential helper 
 (or use SSH)!

Slide 35

Slide 35 text

Gotchas No line ending conversions 
 on Git LFS files! FirstLine SecondLine FirstLine▯Second Line
 Windows macOS

Slide 36

Slide 36 text

Gotchas $ git Your branch is up-to-date with 'origin/master'. Encountered 2 file(s) that should have been pointers, but weren't: path/to/some/file.PNG other/image.PNG

Slide 37

Slide 37 text

Administrator 38 MANAGING GIT REPOS

Slide 38

Slide 38 text

Setup Git LFS 
 on dev machines! Enterprise Config for Git
 https://autode.sk/git

Slide 39

Slide 39 text

Distribute recent Git / Git LFS versions

Slide 40

Slide 40 text

Configure a file size limit on your Git server!

Slide 41

Slide 41 text

Takeaways • Git LFS works at scale • Use smart tracking patterns • Speed up clones with `git lfs clone` • Reject large files on the Git server Questions? @kit3bus