Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
Internals of GitLab and libgit2
Search
Minqi Pan
November 03, 2015
Programming
4
1.8k
Internals of GitLab and libgit2
Minqi Pan
November 03, 2015
Tweet
Share
More Decks by Minqi Pan
See All by Minqi Pan
RubyConf 2017: Packing your Ruby application into a single executable
pmq20
1
640
Node 8 and Friends
pmq20
0
220
Moving Node.js and nodec to v8 Ignition
pmq20
0
330
QCon 2017 Beijing Observations
pmq20
0
250
Node.js Compiler: compiling your Node.js application into a single executable
pmq20
5
25k
Enclose.IO: current cutting-edges and the future work
pmq20
0
220
环境变量为何能让 Ruby 快十倍
pmq20
3
740
Introductions to Node.js internals
pmq20
0
2.1k
How we scaled GitLab for a 30k-employee company
pmq20
6
1.5k
Other Decks in Programming
See All in Programming
プロダクトエンジニアのしごと 〜 受託 × 高難度を乗り越えるOptium開発 〜
algoartis
0
190
一緒に働きたくなるプログラマの思想 #QiitaConference
mu_zaru
80
20k
VitestのIn-Source Testingが便利
taro28
8
2.4k
2ヶ月で生産性2倍、お買い物アプリ「カウシェ」4チーム同時改善の取り組み
ike002jp
1
110
iOSアプリで測る!名古屋駅までの 方向と距離
ryunakayama
0
150
開発者フレンドリーで顧客も満足?Platformの秘密
algoartis
0
190
REALITY コマンド作成チュートリアル
nishiuriraku
0
120
実践Webフロントパフォーマンスチューニング
cp20
45
10k
「理解」を重視したAI活用開発
fast_doctor
0
290
音声プラットフォームのアーキテクチャ変遷から学ぶ、クラウドネイティブなバッチ処理 (20250422_CNDS2025_Batch_Architecture)
thousanda
0
400
UMAPをざっくりと理解 / Overview of UMAP
kaityo256
PRO
3
1.5k
RuboCop: Modularity and AST Insights
koic
2
2.6k
Featured
See All Featured
CSS Pre-Processors: Stylus, Less & Sass
bermonpainter
357
30k
GitHub's CSS Performance
jonrohan
1031
460k
Dealing with People You Can't Stand - Big Design 2015
cassininazir
367
26k
Java REST API Framework Comparison - PWX 2021
mraible
31
8.6k
For a Future-Friendly Web
brad_frost
177
9.7k
Visualizing Your Data: Incorporating Mongo into Loggly Infrastructure
mongodb
45
9.5k
Being A Developer After 40
akosma
91
590k
How to Ace a Technical Interview
jacobian
276
23k
Into the Great Unknown - MozCon
thekraken
38
1.7k
Reflections from 52 weeks, 52 projects
jeffersonlam
349
20k
Fight the Zombie Pattern Library - RWD Summit 2016
marcelosomers
233
17k
What’s in a name? Adding method to the madness
productmarketing
PRO
22
3.4k
Transcript
GitLab 与 libgit2 原理理 Minqi Pan
I’m Minqi Pan github.com/pmq20 twitter @psvr
GitLab 的励志故事 • 2011年年:第⼀一⾏行行代码,基于 Rails 3 • 2012年年:发布 2.1.0~4.0.0 •
2013年年:升级到 Rails 4,发布 6.5.0 在阿姆斯特丹丹注册公司 GitLab B.V. • 2014年年:申请 YC 孵化器成功,发布 7.7.0 获得天使融资12万美⾦金金
GitLab 的励志故事 • 2015年年1⽉月~3⽉月:进驻旧⾦金金⼭山湾区⼭山景城 • 2015年年7⽉月:完成 150万 美元 pre-A 轮融资
• 2015年年9⽉月:完成 570万 美元 A 轮融资 • 2015年年10⽉月:发布 8.1.2
⿊黑盒分析 GitLab
⿊黑盒分析 GitLab HTTP 80 SSH 22 SMTP 25/587
⿊黑盒分析 GitLab Redis Postgre SQL File System
⽩白盒分析 GitLab
⽩白盒分析 NGINX OpenSSH Server Postfix Unicorn Sidekiq Gitlab Workhorse libgit2
gitlab-ci gitlab-rails gitlab-shell mattermost omnibus-ctl git
具体场景分析 • 创建项⽬目 • 通过 HTTP 提交代码 • 通过 SSH
克隆隆项⽬目 • 在⻚页⾯面上查看提交历史
创建项⽬目 浏览器 POST /projects ProjectsController#create CreateService#execute Project#save Postgre SQL NGINX
Unicorn
浏览器 POST /projects … Gitlab::Shell#add_repository GitlabProjects#exec GitlabProjects#add_project $ git \
--git-dir=/home/git/repositories/psvr/repo903.git \ init \ --bare $ /opt/gitlab/embedded/ service/gitlab-shell/ bin/gitlab-projects \ add-project \ psvr/repo903.git FileUtils.ln_sf(“/home/git/gitlab-shell/hooks", "/home/git/repositories/psvr/repo903.git/hooks")
浏览器 POST /projects CreateService#after_create_actions Gitlab::Shell#add_repository ProjectsController#create CreateService#execute Project#save 浏览器 302
重定向
通过 HTTP 提交代码 git push GET http://.../pmq2001/repo903.git/info/ refs?service=git-receive-pack NGINX Gitlab
Workhorse Unicorn func (h *gitHandler) doAuthRequest(r *http.Request) (result *http.Response, err error) GET http://.../pmq2001/repo903.git/ info/refs?service=git-receive-pack 去掉 body
通过 HTTP 提交代码 Unicorn git push … Grack::Auth#auth! Gitlab Workhorse
validated the client request additional request metadata in JSON GET http://.../pmq2001/repo903.git/info/ refs?service=git-receive-pack
通过 HTTP 提交代码 git push GET http://.../pmq2001/repo903.git/info/ refs?service=git-receive-pack … Gitlab
Workhorse $ git \ receive-pack \ --stateless-rpc \ --advertise-refs \ /home/git/repositories/psvr/repo903.git func handleGetInfoRefs (w http.ResponseWriter, r *gitRequest, _ string)
通过 HTTP 提交代码 git push GET http://.../pmq2001/repo903.git/info/ refs?service=git-receive-pack … Gitlab
Workhorse
通过 HTTP 提交代码 git push POST http://.../pmq2001/repo903.git/git- receive-pack NGINX Gitlab
Workhorse Unicorn func (h *gitHandler) doAuthRequest(r *http.Request) (result *http.Response, err error) POST /pmq2001/repo903.git/git-receive- pack 去掉 body
通过 HTTP 提交代码 git push POST http://.../pmq2001/repo903.git/git- receive-pack NGINX Gitlab
Workhorse func handlePostRPC (w http.ResponseWriter, r *gitRequest, rpc string) $ git \ receive-pack \ --stateless-rpc \ /home/git/repositories/psvr/repo903.git
通过 HTTP 提交代码 git push POST http://.../pmq2001/repo903.git/git- receive-pack NGINX Gitlab
Workhorse func handlePostRPC (w http.ResponseWriter, r *gitRequest, rpc string) $ git \ receive-pack \ --stateless-rpc \ /home/git/repositories/psvr/repo903.git $ hooks/pre-receive gitlab-shell
通过 HTTP 提交代码 git push POST http://.../pmq2001/repo903.git/git- receive-pack $ hooks/pre-receive
GitlabAccess#exec … Gitlab Workhorse Unicorn POST "/api/v3/internal/allowed"
通过 HTTP 提交代码 git push POST http://.../pmq2001/repo903.git/git- receive-pack … Unicorn
$ hooks/post-receive Postgre SQL GitlabPostReceive#update_redis Sidekiq PostReceive
通过 HTTP 提交代码 git push POST http://.../pmq2001/repo903.git/git- receive-pack … Unicorn
$ hooks/post-receive GitlabNet#broadcast_message GET /api/v3/internal/broadcast_message Unicorn ProjectCacheWorker Postgre SQL Sidekiq
通过 HTTP 提交代码 git push POST http://.../pmq2001/repo903.git/git- receive-pack NGINX Gitlab
Workhorse
通过 SSH 克隆隆项⽬目 ssh git@gitlab.server OpenSSH Server gitlab-shell GitlabShell#exec
通过 SSH 克隆隆项⽬目 ssh git@gitlab.server GitlabShell#exec [root@i-jl15ph2j ~]# ssh git@gitlab.server
Warning: Permanently added 'gitlab.server,1.2.3.4' (RSA) to the list of known hosts. PTY allocation request failed on channel 0 Welcome to GitLab, psvr! Connection to gitlab.server closed. gitlab-shell OpenSSH Server
通过 SSH 克隆隆项⽬目 git clone GitlabShell#verify_access gitlab-shell OpenSSH Server git@gitlab.server:psvr/repo903.git
Unicorn POST "/api/v3/internal/allowed"
通过 SSH 克隆隆项⽬目 git clone GitlabShell#verify_access gitlab-shell OpenSSH Server git@gitlab.server:psvr/repo903.git
$ git-upload-pack \ /var/opt/gitlab/git-data/repositories/psvr/repo903.git 009bd9be3108e6e60b9954fec57f6ca00d4f74 95170d HEADmulti_ack thin-pack side- band side-band-64k ofs-delta shallow no-progress include-tag multi_ack_detailed 003fd9be3108e6e60b9954fec57f6ca00d4f74 95170d refs/heads/master 0000
在⻚页⾯面上查看提交历史 浏览器 GET Projects::CommitsController#show Repository#commits NGINX Unicorn /psvr/repo903/commits/master gem -
gitlab_git
在⻚页⾯面上查看提交历史 浏览器 GET /psvr/repo903/commits/master gem - gitlab_git … Gitlab::Git::Repository#log $
git \ --git-dir=home/git/repositories/psvr/repo903.git \ -n 10 --format=%H --skip=100 --follow --no-merges
在⻚页⾯面上查看提交历史 浏览器 GET /psvr/repo903/commits/master gem - gitlab_git … Gitlab::Git::Repository#log $
git \ --git-dir=home/git/repositories/psvr/repo903.git \ -n 10 --format=%H --skip=100 --follow --no-merges
在⻚页⾯面上查看提交历史 浏览器 GET /psvr/repo903/commits/master gem - rugged … Rugged::Commit#message rb_git_commit_message_GET
git_commit_message libgit2
libgit2 的特点 portable, pure C, re-entrant, linkable, solid API 提⽰示:对比
ruby 与 mruby
libgit2 是封装良好的 • ⼤大部分头⽂文件位于 src/*.h,是不公开的 • 像最基本的 git_repository 结构体都只有外部声明 •
include/git2/*.h 分类良好,快速定位接⼝口,包含 丰富的注释(对比 mruby =.=) • [ demo ]
libgit2 是抽象良好的 • git2/sys/*_backend.h 解耦, 但是 index 没有解耦 • 通过这些函数挂接新的后端
None
Thank you https://github.com/pmq20/