GitHub Enterprise Integration with AWS

Slide 1

Slide 1 text

GitHub Enterprise How COOKPAD uses GitHub Enterprise to build COOKPAD  Integration with AWS

Slide 2

Slide 2 text

Naoto Takai Cowboy Coder

Slide 3

Slide 3 text

ghe-cleanup-repos

Slide 4

Slide 4 text

We’re Hiring to make everyday cooking fun.

Slide 5

Slide 5 text

20M UU/month 1.3M recipes

Slide 6

Slide 6 text

10 deploys/day 4.4K pull reqs

Slide 7

Slide 7 text

✓Intel Core i7 2600K ✓16 GB Memory ✓80 licenses ✓300+ repositories Github Enterprise VMWare ESXi Helper NAS +

Slide 8

Slide 8 text

Before GitHub Enterprise era...

Slide 9

Slide 9 text

We used Git with SSH Developer Git Server CI Server App Server 1. push 2. pull 3. tag 4. pull 5. deploy LAN AWS

Slide 10

Slide 10 text

We does not use github.com for main project ‣Our security policy ‣Concerns about service availability

Slide 11

Slide 11 text

✗ Our code base is growing ✗ Hard to track changes in the code base ✗ Review was very hard (w/ Review Board)

Slide 12

Slide 12 text

No content

Slide 13

Slide 13 text

We are using GitHub Enterprise since April 2012 but it was a bit diﬃcult for our environments...

Slide 14

Slide 14 text

Our servers are on AWS, but GitHub Enterprise does not work on it. ‣Need to communicate with the servers on AWS such as LDAP, SMTP, etc... ‣Need to access from external network. ‣Without changing deploy process

Slide 15

Slide 15 text

Slide 16

Slide 16 text

GitHub Accessing AWS: LAN AWS LDAP SMTP

Slide 17

Slide 17 text

GitHub Accessing AWS: LAN AWS DIRECT ACCESS LDAP SMTP

Slide 18

Slide 18 text

GitHub LDAP SMTP Accessing AWS: LAN AWS Helper Gateway SSH Tunnel (autossh) GitHub Enterprise does not support SSH tunneling

Slide 19

Slide 19 text

Slide 20

Slide 20 text

Access from External: LAN AWS GitHub Internet Developer Developer

Slide 21

Slide 21 text

Access from External: LAN AWS GitHub Internet Developer Developer DIRECT ACCESS

Slide 22

Slide 22 text

Developer Access from External: LAN AWS GitHub Gateway SSH Tunnel (autossh) Reverse Proxy Internet Developer Tips ‣ DNS lookup Helper

Slide 23

Slide 23 text

Developer Access from External (SSH): LAN AWS GitHub Gateway SSH Tunnel (autossh) Internet Developer Tips: SSH con g hack Host github-enterprise Hostname gateway CheckHostIP no StrictHostKeyChecking no port 2222 Helper

Slide 24

Slide 24 text

Slide 25

Slide 25 text

Developer Git Server CI Server App Server 1. push 2. pull 3. tag 4. pull 5. deploy LAN AWS Deploy Process (old):

Slide 26

Slide 26 text

Deploy Process (new): Developer GitHub Gateway SSH Tunnel (autossh) CI Server Helper LAN AWS App Server Git Server 3. pull 4. tag 1. push 2. push 5. pull 6. deploy ✓We can deploy app even if GitHub Enterprise is down. ✓All commits are mirrored in the git server.

Slide 27

Slide 27 text

Commit Mirroring: GitHub Git Server AWS LAN push

Slide 28

Slide 28 text

Commit Mirroring: GitHub Git Server AWS LAN push DIRECT ACCESS

Slide 29

Slide 29 text

Commit Mirroring: GitHub Git Server AWS LAN clone pusher service hook pull push

Slide 30

Slide 30 text

require 'sinatra' require 'json' post '/update' do payload = JSON.parse(params[:payload]) project = payload['repository']['name'] cmd = "cd #{project} " "&& git remote prune origin " + "&& git fetch -q " + "&& git push -q --mirror git-server" `#{cmd}` end