Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Speaker Deck
PRO
Sign in
Sign up
for free
Stretcher
FUJIWARA Shunichiro
PRO
March 22, 2016
Technology
2
5.5k
Stretcher
Shibuya.go#2
FUJIWARA Shunichiro
PRO
March 22, 2016
Tweet
Share
More Decks by FUJIWARA Shunichiro
See All by FUJIWARA Shunichiro
fujiwara3
PRO
4
1.4k
fujiwara3
PRO
1
2.1k
fujiwara3
PRO
1
120
fujiwara3
PRO
22
21k
fujiwara3
PRO
4
2.9k
fujiwara3
PRO
8
10k
fujiwara3
PRO
18
7.8k
fujiwara3
PRO
9
3.6k
fujiwara3
PRO
0
1.6k
Other Decks in Technology
See All in Technology
whitefox_73
0
190
yoku0825
PRO
3
120
miyakemito
1
570
buildersbox
0
190
norioikedo
0
210
nisshii0313
1
140
andysumi
0
160
sat
40
29k
fufuhu
3
140
buildersbox
0
190
tomihisa
1
1.2k
aditya45
2
1.7k
Featured
See All Featured
rocio
155
11k
erikaheidi
14
4.3k
iamctodd
19
2k
rmw
11
810
keavy
107
14k
tenderlove
53
3.5k
mza
80
4.1k
malarkey
192
8.6k
reverentgeek
167
7.2k
sachag
267
17k
notwaldorf
15
1.8k
samlambert
237
10k
Transcript
Stretcher Shibuya.go #2 2016-03-22 @fujiwara
@fujiwara github.com/fujiwara sfujiwara.hatenablog.com ٕज़෦
What's Stretcher github.com/fujiwara/stretcher Consul / Serf ͱ࿈ܞͯ͠ಈ͘σϓϩΠπʔϧ
Inspired by github.com/sorah/mamiya & AWS CodeDeploy
ઃܭํ AWS͡Όͳͯ͘ಈ͘ tar & rsync & ίϚϯυ࣮ߦ Consul / Serf
ͱૄ݁߹ GoͰॻ͘ Ұͭͷ͜ͱΛ͏·͘Δ
Architecture
ಋೖ࣮ ʙ100 ιʔγϟϧήʔϜɺࣗࣾαʔϏεʙ10(λΠτϧ) ʙ100 ΛՃ͢Δ minne ͷٕज़ج൫ઓུ www.slideshare.net/hsbt/minne-54244702
StretcherͷΔ͜ͱ(1) 1. Manifest URLΛඪ४ೖྗ͔ΒಡΉ JSON(Consul Event) ·ͨ text 1ߦ(SerfͳͲ) 2.
ManifestΛऔಘ s3 or http(s) or file scheme 3. Manifest(YAML)ʹैͬͯDeployॲཧ download & extract & rsync & command
Manifest? src: s3://example.com/app.tar.gz checksum: e0840daaa97cd2cf2175f9e5d133ffb3324a2b93 dest: /home/stretcher/app commands: pre: -
/path/to/notify_start post: - /path/to/restart_app success: - /path/to/notify_ok failure: - /path/to/notify_ng excludes: - "*.pid" - "*.socket"
StretcherͷΔ͜ͱ(2) 1. Manifestʹهड़͞Εͨ src(tar)Λऔಘ 2. pre ίϚϯυΛ࣮ߦ 3. tarΛςϯϙϥϦσΟϨΫτϦʹల։ 4.
dest σΟϨΫτϦʹ rsync 5. post ίϚϯυΛ࣮ߦ 6. success or failure ίϚϯυΛ࣮ߦ
StretcherͷΒͳ͍͜ͱ build, tar, S3ͷΞοϓϩʔυ, consul eventൃߦͳͲ֎ github.com/pepabo/capistrano-stretcher Rundeckͱ͏ྫ kikumoto.hatenablog.com/entry/2015/09/16/190036
Imprementation
Imprementation ͓͜ͱΘΓ: εϥΠυͷίʔυΤϥʔॲཧͷলུଟΊͰ͢
نײ ຊମ700ߦɺςετؚΊͯ1200ߦఔͷখ͞͞ 71 cmd/stretcher/main.go 120 aws.go 66 aws_test.go 115 command.go
101 command_test.go 31 consul.go 62 consul_test.go 212 manifest.go 245 manifest_test.go 171 stretcher.go ------------------------------------------------- 720 total 474 total
go-latestͰ࠷৽όʔδϣϯผ $ stretcher -v version: v0.4.1 build: 2016-02-15T18:36:48+0900 0.4.1 is
not latest, you should upgrade to 0.4.2
go-latestͰ࠷৽όʔδϣϯผ github.com/tcnksm/go-latest func checkLatest(version string) { version = fixVersionStr(version) githubTag
:= &latest.GithubTag{ Owner: "fujiwara", Repository: "stretcher", FixVersionStrFunc: fixVersionStr, } res, err := latest.Check(githubTag, version) if err != nil { fmt.Println(err) return } if res.Outdated { fmt.Printf("%s is not latest, you should upgrade to %s\n", version, res.Current) } }
go-latestͰ࠷৽όʔδϣϯผ • GithubTag - GitHub্ͷλάͱൺֱ • JSON - JSON APIͷϨεϙϯεͱൺֱ
{ "version":"1.2.3", "message":"Fix crash bug when XXX" } • HTMLMeta - URLͰఏڙ͞ΕΔHTMLͷϝλλάͱൺֱ <meta name="go-latest" content="stretcher 0.4.2 ...">
versionͷΛbuild࣌ʹຒΊࠐΉ Makefile GIT_VER := $(shell git describe --tags) packages: gox
(ུ) -ldflags "-X main.version ${GIT_VER}" Gitͷtag͕ main.version ͷͱͯ͠ઃఆ͞ΕΔ
math/rand ͷSeedΛ͍͍ײ͡ʹॳظԽ͢Δ ίϚϯυϥΠϯΦϓγϣϯ -random-sleep ͰదʹSleep ىಈλΠϛϯάΛࢄΒ͢ import crand "crypto/rand" import
mrand "math/rand" // http://stackoverflow.com/questions/6181260/generating-random-numbers-in-go var s int64 if err := binary.Read(crand.Reader, binary.LittleEndian, &s); err != nil { s = time.Now().UnixNano() } mrand.Seed(s)
ඪ४ೖྗͷParse • ConsulͰJSON(ྻܗࣜ)͕͞ΕΔ • SerfͰվߦऴςΩετ͕͞ΕΔ ࣗͰશ෦ಡ·ͳ͍Ͱɺ͍͍ײ͡ʹผ͍ͨ͠
ඪ४ೖྗ͔ΒJSON ·ͨ 1ߦΛಡΉ bufio.Reader.Peek() Ͱઌ಄Λ͍ͯಡΈ͚ func parseEvents() (string, error) {
reader := bufio.NewReader(os.Stdin) b, _ := reader.Peek(1) if string(b) == "[" { ev, err := ParseConsulEvents(reader) if err != nil { return "", err } return string(ev.Payload), nil } else { line, _, err := reader.ReadLine() return string(line), err } }
Consul event JSON PayloadʹBase64͞Ε͕ͨೖ͍ͬͯΔ [ { "ID": "1d6731a8-833c-1aff-94e5-aa5e5a77da9f" "Name": "deploy",
"Payload": "czM6Ly9leGFtcGxlLmNvbS9wYXRoL3RvL3Rhci5neg==", "NodeFilter": "", "ServiceFilter": "", "TagFilter": "", "Version": 1, "LTime": 2, }, ... ]
Consul event JSON ΛಡΉ type ConsulEvents []ConsulEvent type ConsulEvent struct
{ ID string `json:"ID"` Name string `json:"Name"` Payload []byte `json:"Payload"` LTime int `json:"LTime"` } []byte Ͱఆ͓ٛͯ͘͠ͱBase64Λdecodeͯ͘͠ΕΔ
S3, HTTP(S), File͔Βಉ͡Α͏ʹಡΉ Manifestͷ src ʹ4छྨͷURL scheme͕༻Մೳ • s3:// •
http:// • https:// • file:// io.ReadCloser interfaceΛѻ͏Α͏ʹઃܭͯ͠நԽ
S3, HTTP(S), File͔Βಉ͡Α͏ʹಡΉ Manifestͷ src ʹ4छྨͷURL scheme͕༻Մೳ • s3:// •
http:// • https:// • file:// io.ReadCloser interfaceΛѻ͏Α͏ʹઃܭͯ͠நԽ
getURL() - S3, HTTP(S), File͔ΒಡΉ stretcher.getURL() io.ReadCloser, error Λฦ͢
func getURL(urlStr string) (io.ReadCloser, error) { u, _ := url.Parse(urlStr) switch u.Scheme { case "s3": return getS3(u) case "http", "https": return getHTTP(u) case "file": return getFile(u) default: return nil, fmt.Errorf("manifest URL scheme must be s3 or http(s) or file: %s", urlStr) } }
getS3() - S3͔Βऔಘ͢Δ github.com/AdRoll/goamz func getS3(u *url.URL) (io.ReadCloser, error) {
// credentialͷಡΈ͜Έ(ུ) client := s3.New(AWSAuth, AWSRegion) bucket := client.Bucket(u.Host) return bucket.GetReader(u.Path) } // goamz/s3 func (b *Bucket) GetReader(path string) (rc io.ReadCloser, err error) {
getHTTP() - HTTP(S)͔Βऔಘ͢Δ func getHTTP(u *url.URL) (io.ReadCloser, error) { req,
_ := http.NewRequest("GET", u.String(), nil) resp, err := http.DefaultClient.Do(req) if err != nil { return nil, err } return resp.Body, nil }
getFile() - File͔Βऔಘ͢Δ func getFile(u *url.URL) (io.ReadCloser, error) { file,
err := os.Open(u.Path) if err != nil { return nil, err } return file, nil }
srcΛಡΈࠐΜͰϑΝΠϧʹอଘ อଘ͠ͳ͕Β͍ͭͰʹchecksumܭࢉ func (m *Manifest) Deploy() error { src, _
:= getURL(m.Src) defer src.Close() tmp, _ := ioutil.TempFile(os.TempDir(), "stretcher") defer os.Remove(tmp.Name()) written, sum, err := m.copyAndCalcHash(tmp, src)
ϑΝΠϧΛอଘ͠ͳ͕Βchecksumܭࢉ Goຊମͷ io.CopyBuffer() + ϋογϡܭࢉ func (m *Manifest) copyAndCalcHash(dst io.Writer,
src io.Reader) (int64, string, error) { h, _ := m.newHash() // ͨͱ͑ sha256.New() buf := make([]byte, 32*1024) for { nr, er := src.Read(buf) // bufʹಡΈࠐΜͰ if nr > 0 { h.Write(buf[0:nr]) // ϋογϡΛܭࢉ nw, ew := dst.Write(buf[0:nr]) // dstʹॻ͘ if nw > 0 { written += int64(nw) }
if ew != nil { err = ew break }
if nr != nw { err = io.ErrShortWrite break } } if er == io.EOF { break } if er != nil { err = er break } } s := fmt.Sprintf("%x", h.Sum(nil)) return written, s, err }
औಘΛ͍͍ײ͡ʹϩάʹग़͢ Wrote 218,761,853 bytes to /tmp/stretcher900495202 (in 4.187495 sec, 52MB/s)
github.com/dustin/go-humanize begin := time.Now() // URLऔಘͱϑΝΠϧॻ͖ࠐΈ duration := float64(time.Now().Sub(begin).Nanoseconds()) / 1000000000 log.Printf("Wrote %s bytes to %s (in %s sec, %s/s)", humanize.Comma(written), tmp.Name(), humanize.Ftoa(duration), humanize.Bytes(uint64(float64(written)/duration)), )
tar & rsyncͷ࣮ߦ os/exec.Cmd.CombinedOutput() ֎෦ίϚϯυ࣮ߦɺඪ४ग़ྗɺඪ४Τϥʔग़ྗΛ·ͱΊͯฦ͢ out, err := exec.Command("tar", "xf",
tmp.Name()).CombinedOutput()
ϢʔβఆٛίϚϯυ(pre,post)ͷ࣮ߦ sh -c ʹ࣮ͯ͠ߦ out, err := exec.Command("sh", "-c", c.String()).CombinedOutput()
pipe, && || ͳͲ͕ॻ͚ΔͷͰศར ࣮ࡍͷྫ post: - /sbin/service zabbix-agent status || sleep 30 && sudo /sbin/service zabbix-agent start
ϢʔβఆٛίϚϯυ(success, failure)ͷ࣮ߦ stretcherͷ࣮ߦϩά͕ίϚϯυͷstdinʹ͞ΕΔ༷ package stretcher var LogBuffer bytes.Buffer // άϩʔόϧ
func Init() { log.SetOutput(io.MultiWriter(os.Stderr, &LogBuffer)) ඪ४logϞδϡʔϧͷग़ྗઌΛ os.Stderr ͱ LogBuffer ʹ ඪ४Τϥʔग़ྗ͠ͳ͕ΒίϚϯυʹ͢όοϑΝʹཷΊΔ
ϩάΛstdinʹͯ͠ίϚϯυ࣮ߦ buf := bytes.NewBuffer(src.Bytes()) c.InvokePipe(buf) func (c CommandLine) InvokePipe(src io.Reader)
error { cmd := exec.Command("sh", "-c", c.String()) stdin, _ := cmd.StdinPipe() stdout, _ := cmd.StdoutPipe() stderr, _ := cmd.StderrPipe() cmd.Start()
֎෦ίϚϯυͷstdinϩάΛྲྀ͠ࠐΉ stdinΛಡ·ͳ͍߹EPIPE͕ൃੜ͢ΔͷͰແࢹ var wg sync.WaitGroup wg.Add(3) // src => cmd.stdin
go func() { _, err := io.Copy(stdin, src) if e, ok := err.(*os.PathError); ok && e.Err == syscall.EPIPE { // ignore EPIPE } else if err != nil { log.Println("failed to write to STDIN of", c.String(), ":", err) } stdin.Close() wg.Done() }()
֎෦ίϚϯυͷstdout,stderrΛಡΈࠐΉ sync.WaitGroupͰͪ߹Θͤͳ͍ͱඌ͕ΕΔ(͜ͱ͕͋Δ) // cmd.stdout => stretcher.stdout go func() { io.Copy(os.Stdout,
stdout) stdout.Close() wg.Done() }() // cmd.stderr => stretcher.stderr go func() { io.Copy(os.Stderr, stderr) stderr.Close() wg.Done() }() wg.Wait() return cmd.Wait() }
·ͱΊ interfaceΛ͏·͘͏ͱ͖ͬ͢Γͨ͠ίʔυʹ ඪ४Ϟδϡʔϧๅͷࢁ stretcherʹ͍ͭͯͷ࣭ɺ͝ཁ͓ؾܰʹ @fujiwara ·Ͱ