Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Git Graphs, Hashes, and Compression, Oh My

Matthew McCullough
June 17, 2012
Programming
4
1.4k

Git Graphs, Hashes, and Compression, Oh My

Git is a version control system. We can look at it from that high level. Git is a content tracking system. Some teachers advise us to look at it from that lowered elevation. But I will take you to the very bottom. The floor. The code. The algorithms. The directed acyclic graph of hashed bit sequences made efficient through LZW compression and deferred garbage collection determined by node reachability via hash relationships.

“But why?”, you may ask. “Why go this deep?”" Git is a tool that works so well for so many. It mystically corrects anticipated `merge` conflicts. It’s “where did code come from” results from `blame` are impressive. The ability to re-write history through `rebase` is awesome. The globally unique identifier nature of a hash-produced ref is revolutionary.

Uber-geeks are magic-slayers. We want and need to know precisely how things work. Like a hard 50 push-up workout, this study will make working with Git at the daily developer level a fraction of the effort — like a mere ten push-ups. Join Matthew McCullough of GitHub and let’s dig into the guts of Git.

Matthew McCullough

June 17, 2012
Tweet

More Decks by Matthew McCullough

See All by Matthew McCullough
Patterns for Collaborative Software Development in a Social World at JavaOne 2015
matthewmccullough
1
370
Collaborative Software Development in a Social World at the Utah JUG
matthewmccullough
2
150
Git and GitHub Platform Review
matthewmccullough
1
400
Your First Pull Request
matthewmccullough
0
270
Git and GitHub Workflows at the Utah JUG
matthewmccullough
1
770
Agile-ish life at GitHub
matthewmccullough
2
340
Git and GitHub Agile Workflows for HP
matthewmccullough
0
270
Writing Tools on the GitHub Platform
matthewmccullough
0
240
Git Feature Branches as an Art
matthewmccullough
0
150

Other Decks in Programming

See All in Programming
GitHub Actionsのキャッシュと手を挙げることの大切さとそれに必要なこと
satoshi256kbyte
5
430
エンジニアとして関わる要件と仕様(公開用)
murabayashi
0
290
アジャイルを支えるテストアーキテクチャ設計/Test Architecting for Agile
goyoki
9
3.3k
cmp.Or に感動した
otakakot
2
140
距離関数を極める！ / SESSIONS 2024
gam0022
0
280
Amazon Bedrock Agentsを用いてアプリ開発してみた！
har1101
0
330
광고 소재 심사 과정에 AI를 도입하여 광고 서비스 생산성 향상시키기
kakao
PRO
0
170
CSC509 Lecture 12
javiergs
PRO
0
160
Click-free releases & the making of a CLI app
oheyadam
2
110
Duckdb-Wasmでローカルダッシュボードを作ってみた
nkforwork
0
120
最新TCAキャッチアップ
0si43
0
140
どうして僕の作ったクラスが手続き型と言われなきゃいけないんですか
akikogoto
1
120

Featured

See All Featured
Templates, Plugins, & Blocks: Oh My! Creating the theme that thinks of everything
marktimemedia
26
2.1k
Fireside Chat
paigeccino
34
3k
Embracing the Ebb and Flow
colly
84
4.5k
Cheating the UX When There Is Nothing More to Optimize - PixelPioneers
stephaniewalter
280
13k
Making the Leap to Tech Lead
cromwellryan
133
8.9k
Helping Users Find Their Own Way: Creating Modern Search Experiences
danielanewman
29
2.3k
VelocityConf: Rendering Performance Case Studies
addyosmani
325
24k
Principles of Awesome APIs and How to Build Them.
keavy
126
17k
Keith and Marios Guide to Fast Websites
keithpitt
409
22k
Designing the Hi-DPI Web
ddemaree
280
34k
The Power of CSS Pseudo Elements
geoffreycrofte
73
5.3k
Building a Modern Day 
E-commerce SEO Strategy
aleyda
38
6.9k

Transcript

  1. at                  

                                         with  Matthew  McCullough Git Graphs, Hashes, and Compression, Oh My! © 2012, GitHub, Inc.

  2. @matthewmccull [email protected] github.com/training

  3. None
  4. None

  5. GIT - the stupid content tracker "git" can mean anything,

    depending on your mood. * random three-letter combination that is pronounceable, and not actually used by any common UNIX command. The fact that it is a mispronunciation of "get" may or may not be relevant. * stupid. contemptible and despicable. simple. Take your pick from the dictionary of slang. * "global information tracker": you're in a good mood, and it actually works for you. Angels sing, and a light suddenly fills the room. * "goddamn idiotic truckload of sh*t": when it breaks Git is a fast, scalable, distributed revision control system with an unusually rich command set that provides both high-level operations and full access to internals. Git is an Open Source project covered by the GNU General Public License. It was originally written by Linus Torvalds with help of a group of hackers around the net. It is currently maintained by Junio C Hamano. “ ”

  6. 2005 git

  7. Architecture Hashes

  8. centralized VCSs use sequential revision numbers

  9. Git uses a SHA-1 hash

  10. None

  11. 40 hex characters (20 bytes)

  12. 9AB223D28B1AA46EF1780B22F304982E39872C34

  13. 9AB223D28B1AA46EF1780B22F304982E39872C34 <html> <body> <p>This is a test</p> <img src="http://ai.com/icon.gif"> </body>

    </html>

  14. <html> <body> <p>This is a test</p> <img src="http://ai.com/icon.gif"> </body> </html>

    9AB223D28B1AA46EF1780B22F304982E39872C34

  15. Hashed Content Git commits without Git

  16. # Green field project $ git init newproject $ cd

    newproject # ...start coding

  17. .git !"" COMMIT_EDITMSG !"" HEAD !"" MERGE_RR !"" config !""

    description !"" hooks # !"" pre-commit.sample # $"" update.sample !"" index !"" info # $"" exclude !"" logs # !"" HEAD # $"" refs # $"" heads # $"" master !"" objects # !"" 54 # # $"" 3b9bebdc6bd5c4b22136034a95dd097a57d3dd # !"" info # $"" pack !"" refs !"" heads # $"" master $"" tags

  18. $ printf "blob 12\000Hello World\n" | shasum

  19. Architecture Hash Shortcuts

  20. use as little of it as is unique

  21. 9AB223D28B1AA46EF 1780B22F304982E39 872C34 a certain commit

  22. 9AB22F a certain commit

  23. git rev-parse 9AB22F expand the commit ref

  24. Architecture Storage

  25. Typical SCMs use delta storage

  26. CVS / Subversion / darcs / Mercurial

  27. v1 v2 v3 v4 File A File B File C

    File A File B File B File C v5 File A File B File B

  28. v1 v2 v3 v4 File A File B File C

    File A File B File B File C v5 File A File B File B Δ Δ Δ Δ Δ Δ Δ
  29. None

  30. Checkin Checkin Checkin Checkin Checkin Checkin Checkin Checkin Checkin Checkin

    Checkin Delta storage gets slower as the history of a file gets longer

  31. Directed Acyclic Graph

  32. None

  33. Copy of the entire tree per checkin

  34. cp -r srcfolder srcfolder.prev

  35. Why?

  36. v1 v2 v3 v4 File A File B File C

    File A File B File B File C v5 File A File B File B File A File A File C File C File C

  37. v1 v2 v3 v4 File A File B File C

    File A File B File B File C v5 File A File B File B File A File A File C File C File C

  38. hard link to existing identical blobs

  39. v1 v2 v3 v4 File A File B File C

    File A File B File B File C v5 File A File B File B File A File A File C File C File C

  40. v1 v2 v3 v4 File A File B File C

    File A File B File B File C v5 File A File B File B File A File A File C File C File C ß

  41. zlib deflates each blob at commit

  42. v1 v2 v3 v4 File A File B File C

    File A File B File B File C v5 File A File B File B File A File A File C File C File C

  43. v1 v2 v3 v4 File A File B File C

    File A File B File B File C v5 File A File B File B File A File A File C File C File C

  44. zlib deflates the entire repo

  45. v1 v2 v3 v4 File A File B File C

    File A File B File B File C v5 File A File B File B File A File A File C File C File C

  46. v1 v2 v3 v4 File A File B File C

    File A File B File B File C v5 File A File B File B File A File A File C File C File C

  47. 2100 MB became 205 MB Act I

  48. Architecture Hash relationships

  49. ‣Blob ‣Tree ‣Commit ‣Tag

  50. tree tree: 7e8b1 web blob: 9ab16 index.html a10b3 tree blob:

    8d162 logo.jpg blob: 51d22 draw.js 7e8b1 commit tree: a10b3 parent: nil author: Fird committer: Matthew message: Major refactoring of the web content. c67db blob <html> <body></body> </html> 9ab16 blob //Some more javascript var renderSize 51d22 blob 7D 8D B3 7F BD 12 9F E9 7B 78 9D 3F 5C A6 72 CB 8d162

  51. tree tree: 7e8b1 web blob: 9ab16 index.html a10b3 tree blob:

    8d162 logo.jpg blob: 51d22 draw.js 7e8b1 commit tree: a10b3 parent: nil author: Fird committer: Matthew message: Major refactoring of the web content. c67db blob <html> <body></body> </html> 9ab16 blob //Some more javascript var renderSize 51d22 blob 7D 8D B3 7F BD 12 9F E9 7B 78 9D 3F 5C A6 72 CB 8d162

  52. tree tree: 7e8b1 web blob: 9ab16 index.html a10b3 tree blob:

    8d162 logo.jpg blob: 51d22 draw.js 7e8b1 commit tree: a10b3 parent: nil author: Fird committer: Matthew message: Major refactoring of the web content. c67db blob <html> <body></body> </html> 9ab16 blob //Some more javascript var renderSize 51d22 blob 7D 8D B3 7F BD 12 9F E9 7B 78 9D 3F 5C A6 72 CB 8d162

  53. tree tree: 7e8b1 web blob: 9ab16 index.html a10b3 tree blob:

    8d162 logo.jpg blob: 51d22 draw.js 7e8b1 commit tree: a10b3 parent: nil author: Fird committer: Matthew message: Major refactoring of the web content. c67db blob <html> <body></body> </html> 9ab16 blob //Some more javascript var renderSize 51d22 blob 7D 8D B3 7F BD 12 9F E9 7B 78 9D 3F 5C A6 72 CB 8d162

  54. tree tree: 7e8b1 web blob: 9ab16 index.html a10b3 tree blob:

    8d162 logo.jpg blob: 51d22 draw.js 7e8b1 commit tree: a10b3 parent: nil author: Fird committer: Matthew message: Major refactoring of the web content. c67db blob <html> <body></body> </html> 9ab16 blob //Some more javascript var renderSize 51d22 blob 7D 8D B3 7F BD 12 9F E9 7B 78 9D 3F 5C A6 72 CB 8d162

  55. tree tree: 7e8b1 web blob: 9ab16 index.html a10b3 tree blob:

    8d162 logo.jpg blob: 51d22 draw.js 7e8b1 commit tree: a10b3 parent: nil author: Fird committer: Matthew message: Major refactoring of the web content. c67db blob <html> <body></body> </html> 9ab16 blob //Some more javascript var renderSize 51d22 blob 7D 8D B3 7F BD 12 9F E9 7B 78 9D 3F 5C A6 72 CB 8d162

  56. tree tree: 7e8b1 web blob: 9ab16 index.html a10b3 tree blob:

    8d162 logo.jpg blob: 51d22 draw.js 7e8b1 commit tree: a10b3 parent: nil author: Fird committer: Matthew message: Major refactoring of the web content. c67db blob <html> <body></body> </html> 9ab16 blob //Some more javascript var renderSize 51d22 blob 7D 8D B3 7F BD 12 9F E9 7B 78 9D 3F 5C A6 72 CB 8d162

  57. tree tree: 7e8b1 web blob: 9ab16 index.html a10b3 tree blob:

    8d162 logo.jpg blob: 51d22 draw.js 7e8b1 commit tree: a10b3 parent: nil author: Fird committer: Matthew message: Major refactoring of the web content. c67db blob <html> <body></body> </html> 9ab16 blob //Some more javascript var renderSize 51d22 blob 7D 8D B3 7F BD 12 9F E9 7B 78 9D 3F 5C A6 72 CB 8d162

  58. tree tree: 7e8b1 web blob: 9ab16 index.html a10b3 tree blob:

    8d162 logo.jpg blob: 51d22 draw.js 7e8b1 commit tree: a10b3 parent: nil author: Fird committer: Matthew message: Major refactoring of the web content. c67db blob <html> <body></body> </html> 9ab16 blob //Some more javascript var renderSize 51d22 blob 7D 8D B3 7F BD 12 9F E9 7B 78 9D 3F 5C A6 72 CB 8d162

  59. v1 v2 v3 commit tree: 9a87b parent: nil author: Fird

    committer: Matthew message: Major refactoring of the Javascript rendering engine. c67db commit tree: b22c1 parent: c67db author: Tim committer: Fird message: Minor update to HTML 9bd21 commit tree: b22c1 parent: 9bd21 author: Johnny committer: Joe message: New language transations 1c2d7

  60. v1 v2 v3 commit tree: 9a87b parent: nil author: Fird

    committer: Matthew message: Major refactoring of the Javascript rendering engine. c67db commit tree: b22c1 parent: c67db author: Tim committer: Fird message: Minor update to HTML 9bd21 commit tree: b22c1 parent: 9bd21 author: Johnny committer: Joe message: New language transations 1c2d7

  61. v1 v2 v3 commit tree: 9a87b parent: nil author: Fird

    committer: Matthew message: Major refactoring of the Javascript rendering engine. c67db commit tree: b22c1 parent: c67db author: Tim committer: Fird message: Minor update to HTML 9bd21 commit tree: b22c1 parent: 9bd21 author: Johnny committer: Joe message: New language transations 1c2d7

  62. v1 v2 v3 commit tree: 9a87b parent: nil author: Fird

    committer: Matthew message: Major refactoring of the Javascript rendering engine. c67db commit tree: b22c1 parent: c67db author: Tim committer: Fird message: Minor update to HTML 9bd21 commit tree: b22c1 parent: 9bd21 author: Johnny committer: Joe message: New language transations 1c2d7

  63. v1 v2 v3 commit tree: 9a87b parent: nil author: Fird

    committer: Matthew message: Major refactoring of the Javascript rendering engine. c67db commit tree: b22c1 parent: c67db author: Tim committer: Fird message: Minor update to HTML 9bd21 commit tree: b22c1 parent: 9bd21 author: Johnny committer: Joe message: New language transations 1c2d7

  64. v1 v2 v3 commit tree: 9a87b parent: nil author: Fird

    committer: Matthew message: Major refactoring of the Javascript rendering engine. c67db commit tree: b22c1 parent: c67db author: Tim committer: Fird message: Minor update to HTML 9bd21 commit tree: b22c1 parent: 9bd21 author: Johnny committer: Joe message: New language transations 1c2d7

  65. v1 v2 v3 commit tree: 9a87b parent: nil author: Fird

    committer: Matthew message: Major refactoring of the Javascript rendering engine. c67db commit tree: b22c1 parent: c67db author: Tim committer: Fird message: Minor update to HTML 9bd21 commit tree: b22c1 parent: 9bd21 author: Johnny committer: Joe message: New language transations 1c2d7

  66. v1 v2 v3 commit tree: 9a87b parent: nil author: Fird

    committer: Matthew message: Major refactoring of the Javascript rendering engine. c67db commit tree: b22c1 parent: c67db author: Tim committer: Fird message: Minor update to HTML 9bd21 commit tree: b22c1 parent: 9bd21 author: Johnny committer: Joe message: New language transations 1c2d7

  67. RELEASE_1.0 HEAD bug979branch commit c67db commit 9bd21 commit 1c2d7 commit

    8c2d1 commit 1bdcd commit 2daa1

  68. RELEASE_1.0 HEAD bug979branch commit c67db commit 9bd21 commit 1c2d7 commit

    8c2d1 commit 1bdcd commit 2daa1

  69. RELEASE_1.0 HEAD bug979branch commit c67db commit 9bd21 commit 1c2d7 commit

    8c2d1 commit 1bdcd commit 2daa1

  70. RELEASE_1.0 HEAD bug979branch commit c67db commit 9bd21 commit 1c2d7 commit

    8c2d1 commit 1bdcd commit 2daa1

  71. RELEASE_1.0 HEAD bug979branch commit c67db commit 9bd21 commit 1c2d7 commit

    8c2d1 commit 1bdcd commit 2daa1

  72. RELEASE_1.0 HEAD bug979branch commit c67db commit 9bd21 commit 1c2d7 commit

    8c2d1 commit 1bdcd commit 2daa1

  73. Architecture Hash shortcuts

  74. commitish & treeish

  75. commitish = shorthand for commit hashes

  76. treeish = shorthand for tree hashes

  77. 9AB22F a certain commit

  78. 9AB22F^ one commit before a certain commit

  79. 9AB22F^^ two commits before a certain commit

  80. 9AB22F~5 five commits before a certain commit

  81. 9AB223..56CD77 between these two commits

  82. HEAD the most recent commit on this branch

  83. HEAD^ one commit before the most recent commit

  84. HEAD~2 two commits before the most recent commit

  85. HEAD..HEAD^^^ between the given recent commits

  86. master the most recent commit on this branch

  87. master^^ two commits before the most recent commit on this

    branch

  88. master~5 five commits before the most recent commit on this

    branch

  89. remotes/origin/master the most recent commit on this remote tracking branch

  90. origin/master the most recent commit on this remote tracking branch

  91. The Graph Verification

  92. git fsck

  93. git verify-pack -v .git/objects/pack/FILENAME

  94. The Graph More graph navigation

  95. master^{tree} find this commit’s tree

  96. git describe HASH find nearest tag

  97. :/some words commit message that starts with

  98. REF:FILE blob spec

  99. :0:FILE index

  100. :1:FILE common ancestor

  101. :2:FILE target

  102. :3:FILE merging in

  103. @matthewmccull [email protected] github.com/training

  104. at                  

                                         with  Matthew  McCullough Git Graphs, Hashes, and Compression, Oh My! © 2012, GitHub, Inc.