Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Git Graphs, Hashes, and Compression, Oh My

Matthew McCullough
June 17, 2012
Programming
4
1.4k

Git Graphs, Hashes, and Compression, Oh My

Git is a version control system. We can look at it from that high level. Git is a content tracking system. Some teachers advise us to look at it from that lowered elevation. But I will take you to the very bottom. The floor. The code. The algorithms. The directed acyclic graph of hashed bit sequences made efficient through LZW compression and deferred garbage collection determined by node reachability via hash relationships.

“But why?”, you may ask. “Why go this deep?”" Git is a tool that works so well for so many. It mystically corrects anticipated `merge` conflicts. It’s “where did code come from” results from `blame` are impressive. The ability to re-write history through `rebase` is awesome. The globally unique identifier nature of a hash-produced ref is revolutionary.

Uber-geeks are magic-slayers. We want and need to know precisely how things work. Like a hard 50 push-up workout, this study will make working with Git at the daily developer level a fraction of the effort — like a mere ten push-ups. Join Matthew McCullough of GitHub and let’s dig into the guts of Git.

Matthew McCullough

June 17, 2012
Tweet

More Decks by Matthew McCullough

See All by Matthew McCullough
Patterns for Collaborative Software Development in a Social World at JavaOne 2015
matthewmccullough
1
360
Collaborative Software Development in a Social World at the Utah JUG
matthewmccullough
2
140
Git and GitHub Platform Review
matthewmccullough
1
320
Your First Pull Request
matthewmccullough
0
260
Git and GitHub Workflows at the Utah JUG
matthewmccullough
1
720
Agile-ish life at GitHub
matthewmccullough
2
310
Git and GitHub Agile Workflows for HP
matthewmccullough
0
250
Writing Tools on the GitHub Platform
matthewmccullough
0
220
Git Feature Branches as an Art
matthewmccullough
0
140

Other Decks in Programming

See All in Programming
SwiftUI Performance 不要なViewの再描画と更新を抑える
bigamitiongit
1
160
今の SmartHR にエンジニアで入社するとどうなるの？
daisukeshinoku
5
4.6k
PHP8.3の機能を振り返る / Review of PHP 8.3 features
seike460
PRO
1
110
Ruby Function Composition
bkuhlmann
1
330
Blue/Greenデプロイの導入による 運用フローの改善
kudoas
1
350
[技育CAMPアカデミア]アイディアを形に！【超入門】スマホアプリ開発〜リリースまでの流れをご紹介
teamlab
PRO
0
350
⼤規模⾔語モデルの拡張（RAG）が 終わったかも知れない件について
nearme_tech
22
15k
Rubyでたのしむクリエイティブコーディング/Enjoy Creative coding with Ruby
chobishiba
1
160
From Spring Boot 2 to Spring Boot 3 with Java 21 and Jakarta EE
ivargrimstad
0
1.1k
Site Reliability Engineering for GMO
pyama86
6
940
AWS Application Composerで始める、 サーバーレスなデータ基盤構築 / 20240406-jawsug-hokuriku-shinkansen
kasacchiful
1
250
Milestoner
bkuhlmann
1
400

Featured

See All Featured
GraphQLとの向き合い方2022年版
quramy
31
12k
Rebuilding a faster, lazier Slack
samanthasiow
72
8.2k
In The Pink: A Labor of Love
frogandcode
138
21k
The Mythical Team-Month
searls
215
42k
Principles of Awesome APIs and How to Build Them.
keavy
120
16k
Java REST API Framework Comparison - PWX 2021
mraible
PRO
18
6.9k
The Invisible Side of Design
smashingmag
294
49k
Optimising Largest Contentful Paint
csswizardry
7
2.3k
Building Applications with DynamoDB
mza
88
5.6k
How To Stay Up To Date on Web Technology
chriscoyier
782
250k
Pencils Down: Stop Designing & Start Developing
hursman
117
11k
Let's Do A Bunch of Simple Stuff to Make Websites Faster
chriscoyier
501
140k

Transcript

  1. at                  

                                         with  Matthew  McCullough Git Graphs, Hashes, and Compression, Oh My! © 2012, GitHub, Inc.

  2. @matthewmccull [email protected] github.com/training

  3. None
  4. None

  5. GIT - the stupid content tracker "git" can mean anything,

    depending on your mood. * random three-letter combination that is pronounceable, and not actually used by any common UNIX command. The fact that it is a mispronunciation of "get" may or may not be relevant. * stupid. contemptible and despicable. simple. Take your pick from the dictionary of slang. * "global information tracker": you're in a good mood, and it actually works for you. Angels sing, and a light suddenly fills the room. * "goddamn idiotic truckload of sh*t": when it breaks Git is a fast, scalable, distributed revision control system with an unusually rich command set that provides both high-level operations and full access to internals. Git is an Open Source project covered by the GNU General Public License. It was originally written by Linus Torvalds with help of a group of hackers around the net. It is currently maintained by Junio C Hamano. “ ”

  6. 2005 git

  7. Architecture Hashes

  8. centralized VCSs use sequential revision numbers

  9. Git uses a SHA-1 hash

  10. None

  11. 40 hex characters (20 bytes)

  12. 9AB223D28B1AA46EF1780B22F304982E39872C34

  13. 9AB223D28B1AA46EF1780B22F304982E39872C34 <html> <body> <p>This is a test</p> <img src="http://ai.com/icon.gif"> </body>

    </html>

  14. <html> <body> <p>This is a test</p> <img src="http://ai.com/icon.gif"> </body> </html>

    9AB223D28B1AA46EF1780B22F304982E39872C34

  15. Hashed Content Git commits without Git

  16. # Green field project $ git init newproject $ cd

    newproject # ...start coding

  17. .git !"" COMMIT_EDITMSG !"" HEAD !"" MERGE_RR !"" config !""

    description !"" hooks # !"" pre-commit.sample # $"" update.sample !"" index !"" info # $"" exclude !"" logs # !"" HEAD # $"" refs # $"" heads # $"" master !"" objects # !"" 54 # # $"" 3b9bebdc6bd5c4b22136034a95dd097a57d3dd # !"" info # $"" pack !"" refs !"" heads # $"" master $"" tags

  18. $ printf "blob 12\000Hello World\n" | shasum

  19. Architecture Hash Shortcuts

  20. use as little of it as is unique

  21. 9AB223D28B1AA46EF 1780B22F304982E39 872C34 a certain commit

  22. 9AB22F a certain commit

  23. git rev-parse 9AB22F expand the commit ref

  24. Architecture Storage

  25. Typical SCMs use delta storage

  26. CVS / Subversion / darcs / Mercurial

  27. v1 v2 v3 v4 File A File B File C

    File A File B File B File C v5 File A File B File B

  28. v1 v2 v3 v4 File A File B File C

    File A File B File B File C v5 File A File B File B Δ Δ Δ Δ Δ Δ Δ
  29. None

  30. Checkin Checkin Checkin Checkin Checkin Checkin Checkin Checkin Checkin Checkin

    Checkin Delta storage gets slower as the history of a file gets longer

  31. Directed Acyclic Graph

  32. None

  33. Copy of the entire tree per checkin

  34. cp -r srcfolder srcfolder.prev

  35. Why?

  36. v1 v2 v3 v4 File A File B File C

    File A File B File B File C v5 File A File B File B File A File A File C File C File C

  37. v1 v2 v3 v4 File A File B File C

    File A File B File B File C v5 File A File B File B File A File A File C File C File C

  38. hard link to existing identical blobs

  39. v1 v2 v3 v4 File A File B File C

    File A File B File B File C v5 File A File B File B File A File A File C File C File C

  40. v1 v2 v3 v4 File A File B File C

    File A File B File B File C v5 File A File B File B File A File A File C File C File C ß

  41. zlib deflates each blob at commit

  42. v1 v2 v3 v4 File A File B File C

    File A File B File B File C v5 File A File B File B File A File A File C File C File C

  43. v1 v2 v3 v4 File A File B File C

    File A File B File B File C v5 File A File B File B File A File A File C File C File C

  44. zlib deflates the entire repo

  45. v1 v2 v3 v4 File A File B File C

    File A File B File B File C v5 File A File B File B File A File A File C File C File C

  46. v1 v2 v3 v4 File A File B File C

    File A File B File B File C v5 File A File B File B File A File A File C File C File C

  47. 2100 MB became 205 MB Act I

  48. Architecture Hash relationships

  49. ‣Blob ‣Tree ‣Commit ‣Tag

  50. tree tree: 7e8b1 web blob: 9ab16 index.html a10b3 tree blob:

    8d162 logo.jpg blob: 51d22 draw.js 7e8b1 commit tree: a10b3 parent: nil author: Fird committer: Matthew message: Major refactoring of the web content. c67db blob <html> <body></body> </html> 9ab16 blob //Some more javascript var renderSize 51d22 blob 7D 8D B3 7F BD 12 9F E9 7B 78 9D 3F 5C A6 72 CB 8d162

  51. tree tree: 7e8b1 web blob: 9ab16 index.html a10b3 tree blob:

    8d162 logo.jpg blob: 51d22 draw.js 7e8b1 commit tree: a10b3 parent: nil author: Fird committer: Matthew message: Major refactoring of the web content. c67db blob <html> <body></body> </html> 9ab16 blob //Some more javascript var renderSize 51d22 blob 7D 8D B3 7F BD 12 9F E9 7B 78 9D 3F 5C A6 72 CB 8d162

  52. tree tree: 7e8b1 web blob: 9ab16 index.html a10b3 tree blob:

    8d162 logo.jpg blob: 51d22 draw.js 7e8b1 commit tree: a10b3 parent: nil author: Fird committer: Matthew message: Major refactoring of the web content. c67db blob <html> <body></body> </html> 9ab16 blob //Some more javascript var renderSize 51d22 blob 7D 8D B3 7F BD 12 9F E9 7B 78 9D 3F 5C A6 72 CB 8d162

  53. tree tree: 7e8b1 web blob: 9ab16 index.html a10b3 tree blob:

    8d162 logo.jpg blob: 51d22 draw.js 7e8b1 commit tree: a10b3 parent: nil author: Fird committer: Matthew message: Major refactoring of the web content. c67db blob <html> <body></body> </html> 9ab16 blob //Some more javascript var renderSize 51d22 blob 7D 8D B3 7F BD 12 9F E9 7B 78 9D 3F 5C A6 72 CB 8d162

  54. tree tree: 7e8b1 web blob: 9ab16 index.html a10b3 tree blob:

    8d162 logo.jpg blob: 51d22 draw.js 7e8b1 commit tree: a10b3 parent: nil author: Fird committer: Matthew message: Major refactoring of the web content. c67db blob <html> <body></body> </html> 9ab16 blob //Some more javascript var renderSize 51d22 blob 7D 8D B3 7F BD 12 9F E9 7B 78 9D 3F 5C A6 72 CB 8d162

  55. tree tree: 7e8b1 web blob: 9ab16 index.html a10b3 tree blob:

    8d162 logo.jpg blob: 51d22 draw.js 7e8b1 commit tree: a10b3 parent: nil author: Fird committer: Matthew message: Major refactoring of the web content. c67db blob <html> <body></body> </html> 9ab16 blob //Some more javascript var renderSize 51d22 blob 7D 8D B3 7F BD 12 9F E9 7B 78 9D 3F 5C A6 72 CB 8d162

  56. tree tree: 7e8b1 web blob: 9ab16 index.html a10b3 tree blob:

    8d162 logo.jpg blob: 51d22 draw.js 7e8b1 commit tree: a10b3 parent: nil author: Fird committer: Matthew message: Major refactoring of the web content. c67db blob <html> <body></body> </html> 9ab16 blob //Some more javascript var renderSize 51d22 blob 7D 8D B3 7F BD 12 9F E9 7B 78 9D 3F 5C A6 72 CB 8d162

  57. tree tree: 7e8b1 web blob: 9ab16 index.html a10b3 tree blob:

    8d162 logo.jpg blob: 51d22 draw.js 7e8b1 commit tree: a10b3 parent: nil author: Fird committer: Matthew message: Major refactoring of the web content. c67db blob <html> <body></body> </html> 9ab16 blob //Some more javascript var renderSize 51d22 blob 7D 8D B3 7F BD 12 9F E9 7B 78 9D 3F 5C A6 72 CB 8d162

  58. tree tree: 7e8b1 web blob: 9ab16 index.html a10b3 tree blob:

    8d162 logo.jpg blob: 51d22 draw.js 7e8b1 commit tree: a10b3 parent: nil author: Fird committer: Matthew message: Major refactoring of the web content. c67db blob <html> <body></body> </html> 9ab16 blob //Some more javascript var renderSize 51d22 blob 7D 8D B3 7F BD 12 9F E9 7B 78 9D 3F 5C A6 72 CB 8d162

  59. v1 v2 v3 commit tree: 9a87b parent: nil author: Fird

    committer: Matthew message: Major refactoring of the Javascript rendering engine. c67db commit tree: b22c1 parent: c67db author: Tim committer: Fird message: Minor update to HTML 9bd21 commit tree: b22c1 parent: 9bd21 author: Johnny committer: Joe message: New language transations 1c2d7

  60. v1 v2 v3 commit tree: 9a87b parent: nil author: Fird

    committer: Matthew message: Major refactoring of the Javascript rendering engine. c67db commit tree: b22c1 parent: c67db author: Tim committer: Fird message: Minor update to HTML 9bd21 commit tree: b22c1 parent: 9bd21 author: Johnny committer: Joe message: New language transations 1c2d7

  61. v1 v2 v3 commit tree: 9a87b parent: nil author: Fird

    committer: Matthew message: Major refactoring of the Javascript rendering engine. c67db commit tree: b22c1 parent: c67db author: Tim committer: Fird message: Minor update to HTML 9bd21 commit tree: b22c1 parent: 9bd21 author: Johnny committer: Joe message: New language transations 1c2d7

  62. v1 v2 v3 commit tree: 9a87b parent: nil author: Fird

    committer: Matthew message: Major refactoring of the Javascript rendering engine. c67db commit tree: b22c1 parent: c67db author: Tim committer: Fird message: Minor update to HTML 9bd21 commit tree: b22c1 parent: 9bd21 author: Johnny committer: Joe message: New language transations 1c2d7

  63. v1 v2 v3 commit tree: 9a87b parent: nil author: Fird

    committer: Matthew message: Major refactoring of the Javascript rendering engine. c67db commit tree: b22c1 parent: c67db author: Tim committer: Fird message: Minor update to HTML 9bd21 commit tree: b22c1 parent: 9bd21 author: Johnny committer: Joe message: New language transations 1c2d7

  64. v1 v2 v3 commit tree: 9a87b parent: nil author: Fird

    committer: Matthew message: Major refactoring of the Javascript rendering engine. c67db commit tree: b22c1 parent: c67db author: Tim committer: Fird message: Minor update to HTML 9bd21 commit tree: b22c1 parent: 9bd21 author: Johnny committer: Joe message: New language transations 1c2d7

  65. v1 v2 v3 commit tree: 9a87b parent: nil author: Fird

    committer: Matthew message: Major refactoring of the Javascript rendering engine. c67db commit tree: b22c1 parent: c67db author: Tim committer: Fird message: Minor update to HTML 9bd21 commit tree: b22c1 parent: 9bd21 author: Johnny committer: Joe message: New language transations 1c2d7

  66. v1 v2 v3 commit tree: 9a87b parent: nil author: Fird

    committer: Matthew message: Major refactoring of the Javascript rendering engine. c67db commit tree: b22c1 parent: c67db author: Tim committer: Fird message: Minor update to HTML 9bd21 commit tree: b22c1 parent: 9bd21 author: Johnny committer: Joe message: New language transations 1c2d7

  67. RELEASE_1.0 HEAD bug979branch commit c67db commit 9bd21 commit 1c2d7 commit

    8c2d1 commit 1bdcd commit 2daa1

  68. RELEASE_1.0 HEAD bug979branch commit c67db commit 9bd21 commit 1c2d7 commit

    8c2d1 commit 1bdcd commit 2daa1

  69. RELEASE_1.0 HEAD bug979branch commit c67db commit 9bd21 commit 1c2d7 commit

    8c2d1 commit 1bdcd commit 2daa1

  70. RELEASE_1.0 HEAD bug979branch commit c67db commit 9bd21 commit 1c2d7 commit

    8c2d1 commit 1bdcd commit 2daa1

  71. RELEASE_1.0 HEAD bug979branch commit c67db commit 9bd21 commit 1c2d7 commit

    8c2d1 commit 1bdcd commit 2daa1

  72. RELEASE_1.0 HEAD bug979branch commit c67db commit 9bd21 commit 1c2d7 commit

    8c2d1 commit 1bdcd commit 2daa1

  73. Architecture Hash shortcuts

  74. commitish & treeish

  75. commitish = shorthand for commit hashes

  76. treeish = shorthand for tree hashes

  77. 9AB22F a certain commit

  78. 9AB22F^ one commit before a certain commit

  79. 9AB22F^^ two commits before a certain commit

  80. 9AB22F~5 five commits before a certain commit

  81. 9AB223..56CD77 between these two commits

  82. HEAD the most recent commit on this branch

  83. HEAD^ one commit before the most recent commit

  84. HEAD~2 two commits before the most recent commit

  85. HEAD..HEAD^^^ between the given recent commits

  86. master the most recent commit on this branch

  87. master^^ two commits before the most recent commit on this

    branch

  88. master~5 five commits before the most recent commit on this

    branch

  89. remotes/origin/master the most recent commit on this remote tracking branch

  90. origin/master the most recent commit on this remote tracking branch

  91. The Graph Verification

  92. git fsck

  93. git verify-pack -v .git/objects/pack/FILENAME

  94. The Graph More graph navigation

  95. master^{tree} find this commit’s tree

  96. git describe HASH find nearest tag

  97. :/some words commit message that starts with

  98. REF:FILE blob spec

  99. :0:FILE index

  100. :1:FILE common ancestor

  101. :2:FILE target

  102. :3:FILE merging in

  103. @matthewmccull [email protected] github.com/training

  104. at                  

                                         with  Matthew  McCullough Git Graphs, Hashes, and Compression, Oh My! © 2012, GitHub, Inc.