Git and GitHub at YAPC:Asia

Transcript

1. Andy Delcambre sadelcambre
3. YAPC:Asia
4. I don’t write Perl
5. I rarely write Perl
6. Total source lines grouped by language: ! ruby: 316613 (98.99%) sh: 2268 (0.71%) ansic: 743 (0.23%) perl: 179 (0.06%) python: 31 (0.01%)
9. DIRECTED ACYCLIC GRAPH
10. $ git log commit b3f13c1 Author: Andy Delcambre Date: Thu Aug 28 08:29 2014 ! Commit 3 ! commit c79e797 Author: Andy Delcambre Date: Thu Aug 28 08:29 2014 ! Commit 2 ! commit 9a42679 Author: Andy Delcambre Date: Thu Aug 28 08:28 2014 ! Initial commit commit b3f13c1 Commit 3 commit c79e797 Commit 2 commit 9a42679 Initial commit HEAD
11. $ git cat-file -p b3f13c1 tree 9e68447 parent c79e797 author Andy Delcambre 1409182168 committer Andy Delcambre 1409182168 ! Commit 3 commit b3f13c1 Commit 3 tree 9e68447 commit c79e797 Commit 2 Parent
12. $ git cat-file -p 9e68447 100644 blob b465ded README.md commit b3f13c1 Commit 3 tree 9e68447 commit c79e797 Commit 2 Parent blob 9e68447 README.md
13. $ git cat-file -p b465ded # Test ! A test repo, for YAPC:Asia. commit b3f13c1 Commit 3 tree 9e68447 commit c79e797 Commit 2 Parent blob 9e68447 README.md # Test ! A test repo, for YAPC:Asia.
14. GitHub
15. Hosting a Git repo is Secret: EASY
16. enzo $ git init --bare demo.git Initialized empty Git repo in /home/adelcambre/git/demo.git/ ! yt $ git remote add origin adelcambre@enzo:git/demo.git yt $ git push origin master Counting objects: 6, done. Delta compression using up to 8 threads. Compressing objects: 100% (2/2), done. Writing objects: 100% (6/6), 473 bytes, done. Total 6 (delta 0), reused 6 (delta 0) To adelcambre@enzo:git/demo.git * [new branch] master -> master ! yt $ git clone adelcambre@enzo:git/demo.git Cloning into 'demo'... remote: Counting objects: 6, done. remote: Compressing objects: 100% (2/2), done. remote: Total 6 (delta 0), reused 0 (delta 0) Receiving objects: 100% (6/6), done.
18. yt $ git remote add -f origin git@github.com:adelcambre/demo.git Updating origin From github.com:adelcambre/demo * [new branch] master -> origin/master ! yt $ git push origin master Counting objects: 6, done. Delta compression using up to 8 threads. Compressing objects: 100% (2/2), done. Writing objects: 100% (6/6), 473 bytes, done. Total 6 (delta 0), reused 6 (delta 0) To https://github.com/adelcambre/demo.git * [new branch] master -> master
21. Hosting a *lot* of Git repos is Not so Secret: HARD
22. > 7 million repositories
23. 1.5 million pushes
24. 100 million pulls & clones
25. 27 fileservers
26. 100 TB of online storage
28. Grit
29. 1st Commit GitHub 1st Commit Grit
30. Ruby wrapper around Git
31. repo = Grit::Repo.new(“grit.git”) ! repo.commits # => [#<Grit::Commit "e80bbd2ce67651aa18e57fb0b43618ad4baf7750">, #<Grit::Commit "91169e1f5fa4de2eaea3f176461f5dc784796769">, #<Grit::Commit "038af8c329ef7c1bae4568b98bd5c58510465493">, #<Grit::Commit "40d3057d09a7a4d61059bca9dca5ae698de58cbe">, #<Grit::Commit "4ea50f4754937bf19461af58ce3b3d24c77311d9">]
32. This works for everything! ! (locally)
33. 27 fileservers
34. Smoke “Grit in the cloud”
35. The Internet Chimney fs102 fs103 fs104 fs105 … fe102 fe103 fe104 fe105 …
36. module Smoke def self.install! ::Grit.send(:remove_const, :Git) ! ::Grit.const_set(:Git, Smoke::Git) end end
37. ls_tree("HEAD").each do |entry| if entry.blob? load_blob(entry.oid) end end
38. Death by Round Trip
39. A condensed (and probably wrong) history of git
40. Bitkeeper
41. Git
42. Shared library?
43. libgit2
44. JGit
45. libgit2 http://github.com/libgit2/libgit2
46. Rugged http://github.com/libgit2/rugged
47. Ruby wrapper around libgit2
48. repo = Rugged::Repository.new("rugged.git") ! repo.last_commit # => #<Rugged::Commit:2227356240 oid: 0277b7df0fea08d5f3fc7177e47527fc24d96a0e>
49. Perl
50. Git-Raw https://metacpan.org/release/Git-Raw
52. GitRPC The New Hotness aka
53. Like Smoke, But Better
54. Caching
55. Pluggable Wire Protocols
56. Encoding Aware
57. GIT(1) Git Manual GIT(1) ! NAME git - the stupid content tracker
58. GIT(1) Git Manual GIT(1) ! NAME git - the stupid content tracker
59. Git doesn’t care about encodings
60. Smoke doesn’t either
61. GitRPC is encoding aware
62. Detect and tag as soon as possible
63. The Switch
64. class Repository def git Smoke::Repo.new(path) end ! def rpc GitRPC.new(path) end end
66. Graphite
67. Graph Every Call Both GitRPC and Smoke
68. 1 Year ago
72. Where are we today?
76. Good Progress But not done yet
77. Finding the Final 0.1%
78. Backscatter
79. Call Counting
80. def git backscatter_measure Smoke::Repo.new(path) end
82. Method Tracing
83. def git backscatter_trace 20 Smoke::Repo.new(path) end
85. Recap
86. Run both concurrently
87. Measure Everything
88. Use tracing to find the last 0.1%
89. Andy Delcambre sadelcambre
90. Further Reading Git for Computer Scientists http://eagain.net/articles/git-for-computer-scientists/ My mom told me git doesn’t scale https://vimeo.com/64716825 How GitHub uses GitHub to build Github http://zachholman.com/talk/how-github-uses-github-to-build-github/