GIT LFS @ LIGHT SPEED - The story of a Git contribution - Lars Schneider Autodesk Taylor Blau GitHub 

@kit3bus Previous version Current version GIT LFS @ LIGHT SPEED 

@kit3bus Recap: Why Git LFS*? * Git Large File Storage = 100 MB video file 

@kit3bus Clone: Git without LFS 0 MB 

@kit3bus Clone: Git without LFS 101 MB master 1 MB 100 MB 

@kit3bus Clone: Git without LFS 201 MB master + 100 MB 1 MB 100 MB 1 MB 100 MB 

@kit3bus Clone: Git without LFS master 301MB + 100 MB + 100 MB 1 MB 100 MB 1 MB 100 MB 1 MB 100 MB 

@kit3bus Clone: Git with LFS 0 MB 

@kit3bus LFS Server Clone: Git with LFS 1MB master 1 MB 100 MB 0 MB 

@kit3bus LFS Server Clone: Git with LFS 1MB master 1 MB 100 MB 1 MB 100 MB 0 MB 0 MB 

@kit3bus LFS Server Clone: Git with LFS 1MB master 1 MB 100 MB 1 MB 1 MB 100 MB 0 MB 0 MB 0 MB 100 MB 

@kit3bus LFS Server Git with LFS master 1 MB 100 MB 1 MB 1 MB 100 MB 0 MB 0 MB 0 MB 100 MB Clone: 1 MB 

@kit3bus LFS Server Git with LFS master 1 MB 100 MB 1 MB 1 MB 100 MB 0 MB 0 MB 0 MB 100 MB Clone: 1 MB + Checkout: 100 MB 

@kit3bus LFS Server Git with LFS master 1 MB 100 MB 1 MB 1 MB 100 MB 0 MB 0 MB 0 MB 100 MB Clone: 1 MB + Checkout: 100 MB 60% bandwidth saving! 

@kit3bus Git Checkout Git 

@kit3bus Git Checkout Git Git LFS Git clean / smudge filter 

@kit3bus Git Git Checkout Git LFS Git LFS 

@kit3bus Git LFS Git LFS Git LFS One-Shot Filter: Many Git LFS invocations Git Checkout Git 

@kit3bus Git LFS Filter Protocol: One Git LFS invocation Git Checkout Git 

@kit3bus Taylor Code Design Decisions Lars Git Contribution Process Git Checkout Git LFS Git 

@kit3bus How to Contribute to Git Core 

@kit3bus Subscribe to the Mailing List Send an email to "[email protected]" with an empty subject line and the words "subscribe git" in the message body. 

@kit3bus Search the Mailing List Archive http://public-inbox.org/git/ 

@kit3bus Write an RFC* Pro Tip:
 Measure 
 performance * Request for Comment 

@kit3bus Mailing List Tips & Tricks Inline replies Text only 

@kit3bus Make Commits $ git checkout -b myfeature master ... feature ... ... tests ... ... docs ... $ git commit maint for bugfixes Read Documentation/CodingGuidelines 

@kit3bus Make Commits area: short description Explain the problem. Explain the solution. Signed-off-by: Lars Schneider Pro Tip:
 Use a spell checker! 

@kit3bus Check Commits make git rebase master -x "make" make test git diff --check master...HEAD ... https://github.com/larsxschneider/git-list-helper prepare-patch.sh master Lazy? 

@kit3bus Check Commits Pro Tip:
 Enable TravisCI 

@kit3bus Prepare Patch Emails git format-patch master From ea25a1834be7e2da36fd00f4abc403a56d6b35bc Mon Sep 17 00:00:00 2001 From: Lars Schneider Date: Mon, 21 Nov 2016 13:44:08 +0100 Subject: [PATCH v1] short description Detailed commit message. Signed-off-by: Lars Schneider --- Hi, message to mailing list! Cheers, Lars apply.c | 2 +- archive.c | 2 +- 2 files changed, 145 insertions(+), 24 deletions(-) diff --git a/apply.c b/apply.c index 2ed808d429..aa7e6e4359 100644 --- a/apply.c +++ b/apply.c @@ -4328,7 +4328,7 @@ static int try_create_file - if (convert_to_working_tree(path, buf, size, &nbuf)) { + if (convert_to_working_tree(path, buf, size, &nbuf, NULL)) { } -- 2.11.0 Commit Message Additional explanations/ cover letter Diff statistics Patch Your Git version Email header 

@kit3bus Prepare Patch Emails prepare-patch.sh master From ea25a1834be7e2da36fd00f4abc403a56d6b35bc Mon Sep 17 00:00:00 2001 From: Lars Schneider Date: Mon, 21 Nov 2016 13:44:08 +0100 Subject: [PATCH v1] short description Detailed commit message. Signed-off-by: Lars Schneider --- Notes: Base Commit: 787f75f056 (master) Diff on Web: https://github.com/larsxschneider/git/commit/a8ec423355 Checkout: git fetch https://... demo && git checkout a8ec423355 apply.c | 2 +- archive.c | 2 +- 2 files changed, 145 insertions(+), 24 deletions(-) diff --git a/apply.c b/apply.c index 2ed808d429..aa7e6e4359 100644 --- a/apply.c +++ b/apply.c @@ -4328,7 +4328,7 @@ static int try_create_file - if (convert_to_working_tree(path, buf, size, &nbuf)) { + if (convert_to_working_tree(path, buf, size, &nbuf, NULL)) { } -- 2.11.0 Additional explanations/ cover letter 

@kit3bus git send-email /patches/* [email protected] git send-email /patches/* [email protected] [email protected] Send Patch Emails Pro Tip:
 Find reviewers
 with git blame prepare-patch.sh master Lazy? 

@kit3bus Wait... 

@kit3bus Respond to Reviews 

@kit3bus Respond to Reviews 1. Find agreement 2. Update commits 3. Check commits 4. Prepare patch emails Pro Tip:
 Make notes https://github.com/larsxschneider/git-list-helper prepare-patch.sh master 2 

@kit3bus Respond to Reviews 1. Find agreement 2. Update commits 3. Check commits 4. Prepare patch emails 5. Send patch emails and CC reviewers Pro Tip:
 Make notes 

@kit3bus v1 Contribution Cycle Take in
 reviews Send patches RFC 

@kit3bus v1 v2 ... Contribution Cycle Take in
 reviews Send patches RFC 

@kit3bus v1 v2 ... v11 Contribution Cycle Take in
 reviews next master Release! Send patches RFC More: Documentation/SubmittingPatches 

@kit3bus Filter Protocol Statistics 1,628 changes 11 iterations 380 emails 6 months Pro Tip:
 Don't give up! 

34 Git LFS Next steps 

38 October, 2016 December, 2016 November, 2016 filter process is merged into git/next Git v2.11 release 

39 (master) $ git checkout my-feature (my-feature) $ ... 

40 (master) $ GIT_TRACE_PACKET=1 git checkout my-feature packet: git> command=smudge packet: git> pathname=foo/bar.dat packet: git> 0000 packet: git> CONTENT packet: git> 0000 packet: git< status=success packet: git< 0000 packet: git< SMUDGED_CONTENT packet: git< 0000 packet: git< 0000 (my-feature) $ ... 

44 3 1 2 

Start small. 45 

Validate quickly. 46 

Iterate later. 47 

49 Integration tests 

50 

51 Integration tests pkt-line protocol 

52 

53 Integration tests pkt-line protocol process filter protocol 

54 

59 for scanner.Scan() { req := scan.Request() switch req.Header[“command”] { case “clean”: case “smudge”: default: } } if err := scan.Err(); err != nil { // ... } 

60 type Request struct { Header map[string]string Payload []byte } 

62 Integration tests pkt-line protocol process filter protocol pkt-line I/O pt. I 

63 

64 type io.Reader interface { Read(p []byte) (n int, err error) } 

65 type Pktline struct { ... } func (p *Pktline) Pkt() ([]byte, error) { // ... } 

72 type PktlineReader struct { ... } func (r *PktlineReader) Read(p []byte) (n int, err error) { for { pkt, err := r.pkt() if err != nil { return n, err } copy(p[n:], pkt) } return n, nil } 

73 for scanner.Scan() { req := scanner.Request() sig := sha256.New() io.Copy(sig, req.Body) fmt.Fprintln(“OID: %x”, sig.Sum(nil)) } 

77 Integration tests pkt-line protocol process filter protocol pkt-line I/O pt. II pkt-line I/O pt. I 

78 

79 type io.Writer interface { Write(p []byte) (n int, err error) } 

87 type PktlineWriter struct { ... } func (w *PktlineWriter) Write(p []byte) (n int, err error) { var n int for n < len(p) { c := min(len(p[n:]), MaxPacketLength) if err := w.writePkt(p[n:c]); err != nil { return n, err } n += c } return n, nil } 

92 func smudge(out io.Writer, oid string) (n int, err error) { path := lfs.LocalMediaPath(oid) f, err := os.Open(path) if err != nil { return 0, err } return io.Copy(out, f) } 

93 Integration tests pkt-line protocol process filter protocol pkt-line I/O pt. II pkt-line I/O pt. I 

! Release! 94 

80x faster. 95 

Next steps? • Promise capability (discussion in git-core) • File-path exchange • ??? 100 

Thank you. 101