Upgrade to Pro — share decks privately, control downloads, hide ads and more …

GitHub Enterprise Integration with AWS

Naoto Takai
January 23, 2013

GitHub Enterprise Integration with AWS

How COOKPAD uses GitHub Enterprise to build COOKPAD.

Naoto Takai

January 23, 2013
Tweet

More Decks by Naoto Takai

Other Decks in Technology

Transcript

  1. GitHub Enterprise
    How COOKPAD uses GitHub Enterprise to build COOKPAD 
    Integration with AWS

    View Slide

  2. Naoto Takai
    Cowboy Coder

    View Slide

  3. ghe-cleanup-repos

    View Slide

  4. We’re Hiring
    to make everyday cooking fun.

    View Slide

  5. 20M UU/month
    1.3M recipes

    View Slide

  6. 10 deploys/day
    4.4K pull reqs

    View Slide

  7. ✓Intel Core i7 2600K
    ✓16 GB Memory
    ✓80 licenses
    ✓300+ repositories
    Github Enterprise
    VMWare ESXi
    Helper
    NAS
    +

    View Slide

  8. Before GitHub Enterprise era...

    View Slide

  9. We used Git with SSH
    Developer
    Git Server CI Server
    App Server
    1. push
    2. pull
    3. tag
    4. pull
    5. deploy
    LAN
    AWS

    View Slide

  10. We does not use github.com
    for main project
    ‣Our security policy
    ‣Concerns about service availability

    View Slide

  11. ✗ Our code base is growing
    ✗ Hard to track changes in the code base
    ✗ Review was very hard (w/ Review Board)

    View Slide

  12. View Slide

  13. We are using GitHub Enterprise
    since April 2012
    but it was a bit difficult for our environments...

    View Slide

  14. Our servers are on AWS,
    but GitHub Enterprise does not work on it.
    ‣Need to communicate with the servers
    on AWS such as LDAP, SMTP, etc...
    ‣Need to access from external network.
    ‣Without changing deploy process

    View Slide

  15. Our servers are on AWS,
    but GitHub Enterprise does not work on it.
    ‣Need to communicate with the servers
    on AWS such as LDAP, SMTP, etc...
    ‣Need to access from external network.
    ‣Without changing deploy process

    View Slide

  16. GitHub
    Accessing AWS:
    LAN
    AWS
    LDAP
    SMTP

    View Slide

  17. GitHub
    Accessing AWS:
    LAN
    AWS DIRECT ACCESS
    LDAP
    SMTP

    View Slide

  18. GitHub
    LDAP
    SMTP
    Accessing AWS:
    LAN
    AWS
    Helper
    Gateway
    SSH Tunnel (autossh)
    GitHub Enterprise does not support SSH tunneling

    View Slide

  19. Our servers are on AWS,
    but GitHub Enterprise does not work on it.
    ‣Need to communicate with the servers
    on AWS such as LDAP, SMTP, etc...
    ‣Need to access from external network.
    ‣Without changing deploy process

    View Slide

  20. Access from External:
    LAN
    AWS
    GitHub
    Internet
    Developer
    Developer

    View Slide

  21. Access from External:
    LAN
    AWS
    GitHub
    Internet
    Developer
    Developer
    DIRECT ACCESS

    View Slide

  22. Developer
    Access from External:
    LAN
    AWS
    GitHub
    Gateway
    SSH Tunnel (autossh)
    Reverse
    Proxy
    Internet
    Developer
    Tips
    ‣ DNS lookup
    Helper

    View Slide

  23. Developer
    Access from External (SSH):
    LAN
    AWS
    GitHub
    Gateway
    SSH Tunnel (autossh)
    Internet
    Developer
    Tips: SSH con g hack
    Host github-enterprise
    Hostname gateway
    CheckHostIP no
    StrictHostKeyChecking no
    port 2222
    Helper

    View Slide

  24. Our servers are on AWS,
    but GitHub Enterprise does not work on it.
    ‣Need to communicate with the servers
    on AWS such as LDAP, SMTP, etc...
    ‣Need to access from external network.
    ‣Without changing deploy process

    View Slide

  25. Developer
    Git Server CI Server
    App Server
    1. push
    2. pull
    3. tag
    4. pull
    5. deploy
    LAN
    AWS
    Deploy Process (old):

    View Slide

  26. Deploy Process (new):
    Developer GitHub
    Gateway
    SSH Tunnel (autossh)
    CI Server
    Helper
    LAN
    AWS
    App Server Git Server
    3. pull
    4. tag
    1. push
    2. push
    5. pull
    6. deploy
    ✓We can deploy app even if GitHub Enterprise is down.
    ✓All commits are mirrored in the git server.

    View Slide

  27. Commit Mirroring:
    GitHub Git Server
    AWS
    LAN
    push

    View Slide

  28. Commit Mirroring:
    GitHub Git Server
    AWS
    LAN
    push
    DIRECT ACCESS

    View Slide

  29. Commit Mirroring:
    GitHub Git Server
    AWS
    LAN
    clone pusher
    service hook
    pull
    push

    View Slide

  30. require 'sinatra'
    require 'json'
    post '/update' do
    payload = JSON.parse(params[:payload])
    project = payload['repository']['name']
    cmd = "cd #{project} "
    "&& git remote prune origin " +
    "&& git fetch -q " +
    "&& git push -q --mirror git-server"
    `#{cmd}`
    end

    View Slide

  31. Bonus: Chrome Extension

    View Slide

  32. Favicon Hack
    http://os0x.github.com/2012/04/16/favicon-on-canvas/

    View Slide

  33. Logo Hack

    View Slide

  34. cross-browser is still problem...
    We want to use “user JavaScript”

    View Slide

  35. We’re Hiring
    to make everyday cooking fun.

    View Slide