GitHub Enterprise Integration with AWS

How COOKPAD uses GitHub Enterprise to build COOKPAD.

Transcript

1. GitHub Enterprise
   How COOKPAD uses GitHub Enterprise to build COOKPAD 
   Integration with AWS
   

   View Slide

2. Naoto Takai
   Cowboy Coder
   

   View Slide

3. ghe-cleanup-repos
   

   View Slide

4. We’re Hiring
   to make everyday cooking fun.
   

   View Slide

5. 20M UU/month
   1.3M recipes
   

   View Slide

6. 10 deploys/day
   4.4K pull reqs
   

   View Slide

7. ✓Intel Core i7 2600K
   ✓16 GB Memory
   ✓80 licenses
   ✓300+ repositories
   Github Enterprise
   VMWare ESXi
   Helper
   NAS
   +
   

   View Slide

8. Before GitHub Enterprise era...
   

   View Slide

9. We used Git with SSH
   Developer
   Git Server CI Server
   App Server
   1. push
   2. pull
   3. tag
   4. pull
   5. deploy
   LAN
   AWS
   

   View Slide

10. We does not use github.com
    for main project
    ‣Our security policy
    ‣Concerns about service availability
    

    View Slide

11. ✗ Our code base is growing
    ✗ Hard to track changes in the code base
    ✗ Review was very hard (w/ Review Board)
    

    View Slide

12. View Slide

13. We are using GitHub Enterprise
    since April 2012
    but it was a bit difficult for our environments...
    

    View Slide

14. Our servers are on AWS,
    but GitHub Enterprise does not work on it.
    ‣Need to communicate with the servers
    on AWS such as LDAP, SMTP, etc...
    ‣Need to access from external network.
    ‣Without changing deploy process
    

    View Slide

15. Our servers are on AWS,
    but GitHub Enterprise does not work on it.
    ‣Need to communicate with the servers
    on AWS such as LDAP, SMTP, etc...
    ‣Need to access from external network.
    ‣Without changing deploy process
    

    View Slide

16. GitHub
    Accessing AWS:
    LAN
    AWS
    LDAP
    SMTP
    

    View Slide

17. GitHub
    Accessing AWS:
    LAN
    AWS DIRECT ACCESS
    LDAP
    SMTP
    

    View Slide

18. GitHub
    LDAP
    SMTP
    Accessing AWS:
    LAN
    AWS
    Helper
    Gateway
    SSH Tunnel (autossh)
    GitHub Enterprise does not support SSH tunneling
    

    View Slide

19. Our servers are on AWS,
    but GitHub Enterprise does not work on it.
    ‣Need to communicate with the servers
    on AWS such as LDAP, SMTP, etc...
    ‣Need to access from external network.
    ‣Without changing deploy process
    

    View Slide

20. Access from External:
    LAN
    AWS
    GitHub
    Internet
    Developer
    Developer
    

    View Slide

21. Access from External:
    LAN
    AWS
    GitHub
    Internet
    Developer
    Developer
    DIRECT ACCESS
    

    View Slide

22. Developer
    Access from External:
    LAN
    AWS
    GitHub
    Gateway
    SSH Tunnel (autossh)
    Reverse
    Proxy
    Internet
    Developer
    Tips
    ‣ DNS lookup
    Helper
    

    View Slide

23. Developer
    Access from External (SSH):
    LAN
    AWS
    GitHub
    Gateway
    SSH Tunnel (autossh)
    Internet
    Developer
    Tips: SSH con g hack
    Host github-enterprise
    Hostname gateway
    CheckHostIP no
    StrictHostKeyChecking no
    port 2222
    Helper
    

    View Slide

24. Our servers are on AWS,
    but GitHub Enterprise does not work on it.
    ‣Need to communicate with the servers
    on AWS such as LDAP, SMTP, etc...
    ‣Need to access from external network.
    ‣Without changing deploy process
    

    View Slide

25. Developer
    Git Server CI Server
    App Server
    1. push
    2. pull
    3. tag
    4. pull
    5. deploy
    LAN
    AWS
    Deploy Process (old):
    

    View Slide

26. Deploy Process (new):
    Developer GitHub
    Gateway
    SSH Tunnel (autossh)
    CI Server
    Helper
    LAN
    AWS
    App Server Git Server
    3. pull
    4. tag
    1. push
    2. push
    5. pull
    6. deploy
    ✓We can deploy app even if GitHub Enterprise is down.
    ✓All commits are mirrored in the git server.
    

    View Slide

27. Commit Mirroring:
    GitHub Git Server
    AWS
    LAN
    push
    

    View Slide

28. Commit Mirroring:
    GitHub Git Server
    AWS
    LAN
    push
    DIRECT ACCESS
    

    View Slide

29. Commit Mirroring:
    GitHub Git Server
    AWS
    LAN
    clone pusher
    service hook
    pull
    push
    

    View Slide

30. require 'sinatra'
    require 'json'
    post '/update' do
    payload = JSON.parse(params[:payload])
    project = payload['repository']['name']
    cmd = "cd #{project} "
    "&& git remote prune origin " +
    "&& git fetch -q " +
    "&& git push -q --mirror git-server"
    `#{cmd}`
    end
    

    View Slide

31. Bonus: Chrome Extension
    

    View Slide

32. Favicon Hack
    http://os0x.github.com/2012/04/16/favicon-on-canvas/
    

    View Slide

33. Logo Hack
    

    View Slide

34. cross-browser is still problem...
    We want to use “user JavaScript”
    

    View Slide

35. We’re Hiring
    to make everyday cooking fun.
    

    View Slide