Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Internals of GitLab and libgit2
Search
Minqi Pan
November 03, 2015
Programming
4
1.7k
Internals of GitLab and libgit2
Minqi Pan
November 03, 2015
Tweet
Share
More Decks by Minqi Pan
See All by Minqi Pan
RubyConf 2017: Packing your Ruby application into a single executable
pmq20
1
490
Node 8 and Friends
pmq20
0
160
Moving Node.js and nodec to v8 Ignition
pmq20
0
250
QCon 2017 Beijing Observations
pmq20
0
190
Node.js Compiler: compiling your Node.js application into a single executable
pmq20
5
24k
Enclose.IO: current cutting-edges and the future work
pmq20
0
170
环境变量为何能让 Ruby 快十倍
pmq20
3
620
Introductions to Node.js internals
pmq20
0
2k
How we scaled GitLab for a 30k-employee company
pmq20
6
1.3k
Other Decks in Programming
See All in Programming
Deep Dive 大規模システムアーキテクチャ/開発組織エンジニアリング / Deep Dive Large-Scale System Architecture, Development Organization Engineering
nrslib
15
2.9k
設計の知識と技能で駆動するソフトウェア開発
masuda220
PRO
18
11k
WinUI 3デモ - "CommunityToolkit.Mvvm"NuGetパッケージ編
andrewkeepcoding
0
140
どうしてこうなった命名集 ~🔥編~ / OOC 2024 LT
pictiny
4
2.9k
【KMC春合宿2024】実装視点で見るNeural Radiance Fields
runningoutrate
0
150
実践!RDRAを活用した既存システムの仕様変更 / Specification Changes in Existing Systems Utilizing RDRA
imamotohikaru
0
2.7k
Go1.22からの疑似乱数生成器について/go-122-pseudo-random-generator
convto
1
160
The Future of C++ Interoperability: Insights from Porting a Game to Swift
teamhimeh
0
280
Parallel Socket Communication in Swift
s_shimotori
0
260
今の SmartHR にエンジニアで入社するとどうなるの?
daisukeshinoku
3
1.5k
Laravel標準バリデーションでできること
hmb_ok
2
360
DocC Tutorial と TCA におけるテスト機能の紹介
kalupas226
1
330
Featured
See All Featured
Product Roadmaps are Hard
iamctodd
43
9.6k
VelocityConf: Rendering Performance Case Studies
addyosmani
319
23k
The Pragmatic Product Professional
lauravandoore
24
5.7k
Creatively Recalculating Your Daily Design Routine
revolveconf
209
11k
How To Stay Up To Date on Web Technology
chriscoyier
781
250k
Web development in the modern age
philhawksworth
201
10k
Building Adaptive Systems
keathley
29
1.8k
Visualization
eitanlees
135
14k
Imperfection Machines: The Place of Print at Facebook
scottboms
257
12k
Why Our Code Smells
bkeepers
PRO
330
56k
GraphQLとの向き合い方2022年版
quramy
28
12k
The Mythical Team-Month
searls
214
42k
Transcript
GitLab 与 libgit2 原理理 Minqi Pan
I’m Minqi Pan github.com/pmq20 twitter @psvr
GitLab 的励志故事 • 2011年年:第⼀一⾏行行代码,基于 Rails 3 • 2012年年:发布 2.1.0~4.0.0 •
2013年年:升级到 Rails 4,发布 6.5.0 在阿姆斯特丹丹注册公司 GitLab B.V. • 2014年年:申请 YC 孵化器成功,发布 7.7.0 获得天使融资12万美⾦金金
GitLab 的励志故事 • 2015年年1⽉月~3⽉月:进驻旧⾦金金⼭山湾区⼭山景城 • 2015年年7⽉月:完成 150万 美元 pre-A 轮融资
• 2015年年9⽉月:完成 570万 美元 A 轮融资 • 2015年年10⽉月:发布 8.1.2
⿊黑盒分析 GitLab
⿊黑盒分析 GitLab HTTP 80 SSH 22 SMTP 25/587
⿊黑盒分析 GitLab Redis Postgre SQL File System
⽩白盒分析 GitLab
⽩白盒分析 NGINX OpenSSH Server Postfix Unicorn Sidekiq Gitlab Workhorse libgit2
gitlab-ci gitlab-rails gitlab-shell mattermost omnibus-ctl git
具体场景分析 • 创建项⽬目 • 通过 HTTP 提交代码 • 通过 SSH
克隆隆项⽬目 • 在⻚页⾯面上查看提交历史
创建项⽬目 浏览器 POST /projects ProjectsController#create CreateService#execute Project#save Postgre SQL NGINX
Unicorn
浏览器 POST /projects … Gitlab::Shell#add_repository GitlabProjects#exec GitlabProjects#add_project $ git \
--git-dir=/home/git/repositories/psvr/repo903.git \ init \ --bare $ /opt/gitlab/embedded/ service/gitlab-shell/ bin/gitlab-projects \ add-project \ psvr/repo903.git FileUtils.ln_sf(“/home/git/gitlab-shell/hooks", "/home/git/repositories/psvr/repo903.git/hooks")
浏览器 POST /projects CreateService#after_create_actions Gitlab::Shell#add_repository ProjectsController#create CreateService#execute Project#save 浏览器 302
重定向
通过 HTTP 提交代码 git push GET http://.../pmq2001/repo903.git/info/ refs?service=git-receive-pack NGINX Gitlab
Workhorse Unicorn func (h *gitHandler) doAuthRequest(r *http.Request) (result *http.Response, err error) GET http://.../pmq2001/repo903.git/ info/refs?service=git-receive-pack 去掉 body
通过 HTTP 提交代码 Unicorn git push … Grack::Auth#auth! Gitlab Workhorse
validated the client request additional request metadata in JSON GET http://.../pmq2001/repo903.git/info/ refs?service=git-receive-pack
通过 HTTP 提交代码 git push GET http://.../pmq2001/repo903.git/info/ refs?service=git-receive-pack … Gitlab
Workhorse $ git \ receive-pack \ --stateless-rpc \ --advertise-refs \ /home/git/repositories/psvr/repo903.git func handleGetInfoRefs (w http.ResponseWriter, r *gitRequest, _ string)
通过 HTTP 提交代码 git push GET http://.../pmq2001/repo903.git/info/ refs?service=git-receive-pack … Gitlab
Workhorse
通过 HTTP 提交代码 git push POST http://.../pmq2001/repo903.git/git- receive-pack NGINX Gitlab
Workhorse Unicorn func (h *gitHandler) doAuthRequest(r *http.Request) (result *http.Response, err error) POST /pmq2001/repo903.git/git-receive- pack 去掉 body
通过 HTTP 提交代码 git push POST http://.../pmq2001/repo903.git/git- receive-pack NGINX Gitlab
Workhorse func handlePostRPC (w http.ResponseWriter, r *gitRequest, rpc string) $ git \ receive-pack \ --stateless-rpc \ /home/git/repositories/psvr/repo903.git
通过 HTTP 提交代码 git push POST http://.../pmq2001/repo903.git/git- receive-pack NGINX Gitlab
Workhorse func handlePostRPC (w http.ResponseWriter, r *gitRequest, rpc string) $ git \ receive-pack \ --stateless-rpc \ /home/git/repositories/psvr/repo903.git $ hooks/pre-receive gitlab-shell
通过 HTTP 提交代码 git push POST http://.../pmq2001/repo903.git/git- receive-pack $ hooks/pre-receive
GitlabAccess#exec … Gitlab Workhorse Unicorn POST "/api/v3/internal/allowed"
通过 HTTP 提交代码 git push POST http://.../pmq2001/repo903.git/git- receive-pack … Unicorn
$ hooks/post-receive Postgre SQL GitlabPostReceive#update_redis Sidekiq PostReceive
通过 HTTP 提交代码 git push POST http://.../pmq2001/repo903.git/git- receive-pack … Unicorn
$ hooks/post-receive GitlabNet#broadcast_message GET /api/v3/internal/broadcast_message Unicorn ProjectCacheWorker Postgre SQL Sidekiq
通过 HTTP 提交代码 git push POST http://.../pmq2001/repo903.git/git- receive-pack NGINX Gitlab
Workhorse
通过 SSH 克隆隆项⽬目 ssh
[email protected]
OpenSSH Server gitlab-shell GitlabShell#exec
通过 SSH 克隆隆项⽬目 ssh
[email protected]
GitlabShell#exec [root@i-jl15ph2j ~]# ssh
[email protected]
Warning: Permanently added 'gitlab.server,1.2.3.4' (RSA) to the list of known hosts. PTY allocation request failed on channel 0 Welcome to GitLab, psvr! Connection to gitlab.server closed. gitlab-shell OpenSSH Server
通过 SSH 克隆隆项⽬目 git clone GitlabShell#verify_access gitlab-shell OpenSSH Server
[email protected]
:psvr/repo903.git
Unicorn POST "/api/v3/internal/allowed"
通过 SSH 克隆隆项⽬目 git clone GitlabShell#verify_access gitlab-shell OpenSSH Server
[email protected]
:psvr/repo903.git
$ git-upload-pack \ /var/opt/gitlab/git-data/repositories/psvr/repo903.git 009bd9be3108e6e60b9954fec57f6ca00d4f74 95170d HEADmulti_ack thin-pack side- band side-band-64k ofs-delta shallow no-progress include-tag multi_ack_detailed 003fd9be3108e6e60b9954fec57f6ca00d4f74 95170d refs/heads/master 0000
在⻚页⾯面上查看提交历史 浏览器 GET Projects::CommitsController#show Repository#commits NGINX Unicorn /psvr/repo903/commits/master gem -
gitlab_git
在⻚页⾯面上查看提交历史 浏览器 GET /psvr/repo903/commits/master gem - gitlab_git … Gitlab::Git::Repository#log $
git \ --git-dir=home/git/repositories/psvr/repo903.git \ -n 10 --format=%H --skip=100 --follow --no-merges
在⻚页⾯面上查看提交历史 浏览器 GET /psvr/repo903/commits/master gem - gitlab_git … Gitlab::Git::Repository#log $
git \ --git-dir=home/git/repositories/psvr/repo903.git \ -n 10 --format=%H --skip=100 --follow --no-merges
在⻚页⾯面上查看提交历史 浏览器 GET /psvr/repo903/commits/master gem - rugged … Rugged::Commit#message rb_git_commit_message_GET
git_commit_message libgit2
libgit2 的特点 portable, pure C, re-entrant, linkable, solid API 提⽰示:对比
ruby 与 mruby
libgit2 是封装良好的 • ⼤大部分头⽂文件位于 src/*.h,是不公开的 • 像最基本的 git_repository 结构体都只有外部声明 •
include/git2/*.h 分类良好,快速定位接⼝口,包含 丰富的注释(对比 mruby =.=) • [ demo ]
libgit2 是抽象良好的 • git2/sys/*_backend.h 解耦, 但是 index 没有解耦 • 通过这些函数挂接新的后端
None
Thank you https://github.com/pmq20/