Building a git reverse proxy in Go for Caddy

Slide 1

Slide 1 text

No content

Slide 2

Slide 2 text

Building a git reverse proxy G f C

Slide 3

Slide 3 text

@brodock blog.gabrielmazetto.eti.br Gabriel Mazetto

Slide 4

Slide 4 text

No content

Slide 5

Slide 5 text

That version control thing...

Slide 6

Slide 6 text

This is a talk about git and its internals f G ...

Slide 7

Slide 7 text

I WILL NOT TEACH YOU HOW TO

Slide 8

Slide 8 text

No content

Slide 9

Slide 9 text

I have to introduce you to a lot of concepts before...

Slide 10

Slide 10 text

$ git help clone Clone a repository into a new directory init Create an empty Git repository or reinitialize an existing one add Add file contents to the index mv Move or rename a file, a directory, or a symlink reset Reset current HEAD to the specified state rm Remove files from the working tree and from the index log Show commit logs show Show various types of objects status Show the working tree status branch List, create, or delete branches checkout Switch branches or restore working tree files commit Record changes to the repository diff Show changes between commits, commit and working tree, etc merge Join two or more development histories together rebase Forward-port local commits to the updated upstream head tag Create, list, delete or verify a tag object signed with GPG fetch Download objects and refs from another repository pull Fetch from and integrate with another repository or a local branch push Update remote refs along with associated objects

Slide 11

Slide 11 text

What git really is?

Slide 12

Slide 12 text

$ find bin/ libexec/ | wc -l 76 not a single binary file...

Slide 13

Slide 13 text

No content

Slide 14

Slide 14 text

Unix Philosophy: S , ,

Slide 15

Slide 15 text

$ find bin/ libexec/ ./bin/git ./bin/git-cvsserver ./bin/git-receive-pack ./bin/git-shell ./bin/git-upload-archive ./bin/git-upload-pack ./libexec/gitcore/git-rebase ./libexec/gitcore/git-stash ./libexec/gitcore/git-submodule ./libexec/gitcore/git-mergetool ./libexec/gitcore/git-bisect ./libexec/gitcore/git-daemon ./libexec/gitcore/git-rebase--am ./libexec/gitcore/git-parse-remote ./libexec/gitcore/git-svn ./libexec/gitcore/git-http-backendu ... perl shell script

Slide 16

Slide 16 text

Where is my code ?

Slide 17

Slide 17 text

P Object Database (ODB)

Slide 18

Slide 18 text

How current ODB backend work?

Slide 19

Slide 19 text

$ git log commit 74ac5c498323b39dfb14eec9244b5f628b717555 Author: Gabriel Mazetto Date: Mon Sep 18 10:49:51 2017 +0200 Added support for Git RPC with upload-pack (clone works now) type SHA1 hash (key) content (data)

Slide 20

Slide 20 text

So is this how the commit object look like?

Slide 21

Slide 21 text

$ git show-ref --head --hash HEAD 74ac5c498323b39dfb14eec9244b5f628b717555 $ git cat-file commit HEAD tree f7530ab4de2105b49789dda745fb450b95c2a33b parent 2d3df47cef64497c77f52f0803e2ca1467f5be1a author Gabriel Mazetto 1505724591 +0200 committer Gabriel Mazetto 1505724591 +0200 Added support for Git RPC with upload-pack (clone works now)

Slide 22

Slide 22 text

$ git cat-file -p f7530ab4de2105b49789dda745fb450b95c2a33b 100644 blob ffd2a4a2e1b6f84ec571a84a4bd6f001643a0745 .gitignore 100644 blob 0929ab8179725c680176ea5001dbab28faea166d README.md 100644 blob 1cdeb2b8f92771a60fe7152f541fdb939f185b86 git.go 100644 blob 7ddbc56e43e5b6ab8e7e00c88589f78c2b7eef04 gitproxy.go 100644 blob 76133a6764e0e49b787dbb49a3103f64f83246f1 setup.go 100644 blob 18b3b795d9a196ecfaf9548be5f721f3f878d9ae setup_test.go

Slide 23

Slide 23 text

Slide 24

Slide 24 text

$ git cat-file 2d3df47cef64497c77f52f0803e2ca1467f5be1a tree 615b472d533b66b908f7d79a0b9d18be32779de4 parent 7fa2356cc5ecd80e949803447dfeb8b566764186 author Gabriel Mazetto 1505720787 +0200 committer Gabriel Mazetto 1505721333 +0200 Added GitAdapter, fixed things and now ls-remote works.

Slide 25

Slide 25 text

74ac5c4 commit f7530ab tree ffd2a4a blob 1cdeb2b blob 2d3df47 commit 615b472 tree eb5bce0 blob ...

Slide 26

Slide 26 text

How is the hash generated?

Slide 27

Slide 27 text

SHA1([obj-type][size][NUL][object-content])

Slide 28

Slide 28 text

$ printf "commit %s\0" $(git cat-file commit HEAD | wc -c) commit 279 NUL byte character

Slide 29

Slide 29 text

$ (printf "commit %s\0" $(git cat-file commit HEAD | wc -c); git cat-file commit HEAD) commit 279tree f7530ab4de2105b49789dda745fb450b95c2a33b parent 2d3df47cef64497c77f52f0803e2ca1467f5be1a author Gabriel Mazetto 1505724591 +0200 committer Gabriel Mazetto 1505724591 +0200 Added support for Git RPC with upload-pack (clone works now)

Slide 30

Slide 30 text

$ (printf "commit %s\0" $(git cat-file commit HEAD | wc -c); git cat-file commit HEAD) | sha1sum 74ac5c498323b39dfb14eec9244b5f628b717555 $ git show-ref --head --hash HEAD 74ac5c498323b39dfb14eec9244b5f628b717555

Slide 31

Slide 31 text

No content

Slide 32

Slide 32 text

Let's talk about protocols...

Slide 33

Slide 33 text

Syncing your

Slide 34

Slide 34 text

git, ssh, https* WebDAV, dumb, smart

Slide 35

Slide 35 text

The git:// protocol $ git daemon --base-path=/srv/git READONLY

Slide 36

Slide 36 text

The ssh:// protocol $ git-upload-pack $ git-receive-pack

Slide 37

Slide 37 text

The webdav:// protocol

Slide 38

Slide 38 text

The dumb protocol $ update-server-info READONLY

Slide 39

Slide 39 text

The smart protocol $ git-upload-pack $ git-receive-pack

Slide 40

Slide 40 text

Let's talk about C

Slide 41

Slide 41 text

Single Binary $ caddy -conf /path/to/Caddyfile

Slide 42

Slide 42 text

It's not only a HTTP

Slide 43

Slide 43 text

Secure by Default! L ' E MITM attack detection

Slide 44

Slide 44 text

label1 { directive1 arg1 directive2 arg1 arg2 directive3 { subdir1 arg1 arg2 subdir2 arg1 arg2 } }

Slide 45

Slide 45 text

https://awesomeapp.example.com { root /srv/awesomeapp.example.com log /var/opt/log/awesomeapp.log { rotate_size 50 # after 50MB rotate_keep 20 # log files to keep rotate_compress # gzip after rotation } proxy / localhost:3000 localhost:3001 { policy round_robin # rotate requests transparent # forward request IPs } }