Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Speaker Deck
PRO
Sign in
Sign up
for free
AWSの「隙間」を埋める隙間家具 OSS 開発 / AWS DevDay Tokyo 2019
FUJIWARA Shunichiro
PRO
October 03, 2019
Technology
8
10k
AWSの「隙間」を埋める隙間家具 OSS 開発 / AWS DevDay Tokyo 2019
FUJIWARA Shunichiro
PRO
October 03, 2019
Tweet
Share
More Decks by FUJIWARA Shunichiro
See All by FUJIWARA Shunichiro
fujiwara3
PRO
4
1.4k
fujiwara3
PRO
1
2k
fujiwara3
PRO
1
120
fujiwara3
PRO
22
21k
fujiwara3
PRO
4
2.9k
fujiwara3
PRO
18
7.8k
fujiwara3
PRO
9
3.6k
fujiwara3
PRO
0
1.6k
fujiwara3
PRO
5
6.3k
Other Decks in Technology
See All in Technology
conciergeu
0
130
miura55
0
240
yasuyukiyamasaki
1
160
lmi
2
850
sat
39
29k
yunoda
0
100
willnet
12
4k
kawaguti
2
410
buildersbox
0
110
subroh0508
4
220
gamella
3
1.4k
myhomenwlab
1
230
Featured
See All Featured
holman
288
130k
pauljervisheath
196
15k
shpigford
369
42k
yeseniaperezcruz
302
31k
shlominoach
176
7.5k
trallard
14
710
shpigford
165
19k
andyhume
64
3.7k
mongodb
23
3.9k
rmw
11
810
morganepeng
93
14k
destraynor
146
19k
Transcript
AWSͷʮ伱ؒʯΛຒΊΔ 伱ؒՈ۩ OSS ։ൃ ౻ݪढ़Ұ @fujiwara
@fujiwara SREνʔϜ github.com/fujiwara sfujiwara.hatenablog.com
Game & Community
Agenda AWSͷʮ伱ؒʯͱ ʮ伱ؒՈ۩ OSSʯͷ࣮ྫͱઃܭࢥ Rin / s32cs / ssmwrap ͳͥ
OSS ͳͷ͔
AWSͷʮ伱ؒʯͱ
AWSͷʮ伱ؒʯͱ ϚωʔδυαʔϏε࠷ॳίΞػೳͰϦϦʔε͞Ε (ཁΛऔΓೖΕͳ͕Β) ঃʑʹػೳ͕૿͍͑ͯ͘
ྫ: RDS for MySQL ͷྺ࢙ 2009-10 ϩʔϯν 2010-05 Multi-AZػೳ 2015-12
ҙλΠϜκʔϯઃఆ 2018-01 CloudWatch Logsʹϩάग़ྗػೳ αʔϏεͷࠜװʹؔΘΔίΞػೳ (ྫ:Multi-AZ) ͘͝ॳظʹ࣮͞ΕΔ͕… ͰԿͱ͔ͳΔػೳ (ྫ:λΠϜκʔϯ) ଞαʔϏεͱͷ࿈ܞศརػೳ (ྫ:Logsग़ྗ) ޙճ͠ʹ͞Ε͕ͪ ← 伱ؒ
ͰԿͱ͔ͳΔػೳޙճ͠ʹͳΓ͕ͪ ྫ͑λΠϜκʔϯΛ JST ʹઃఆ͍ͨ͠߹ ύϥϝʔλάϧʔϓͷ init_connect ͰҎԼͷ SQL Λࢦఆ1 SET
SESSION time_zone = CASE WHEN POSITION('rds' IN CURRENT_USER()) = 1 THEN 'UTC' ELSE 'Asia/Tokyo' END; rds ͔Β࢝·ΔϢʔβʔ(RDS෦Ͱར༻͢Δͷ)Ͱ UTC ͦΕҎ֎ͷϢʔβʔʹ Asia/Tokyo (JST) ʹઃఆ͢Δ Bad Know-how... 1 https://qiita.com/j3tm0t0/items/089ef96ba131df079ca4
ଞͷαʔϏεͱͷ࿈ܞศརػೳ… RDS ͷϩάΛ֎෦ʹྲྀ͔ͨͬͨ͠ github.com/kenjiskywalker/fluent-plugin-rds-slowlog github.com/shinsaka/fluent-plugin-rds-log github.com/acidlemon/rds-throwlog ΈΜͳؤு͍ͬͯͨ CloudWatch Logs ࿈ܞϦϦʔεͰઃఆϙνͰྃ͢ΔΑ͏ʹ
伱͕ؒଟ͍ϚωʔδυαʔϏεΛ͏͔Ͳ͏͔ ϚωʔδυαʔϏεউखʹڧ͘ͳΔ ࣗલӡ༻উखʹڧ͘ͳΒͳ͍ ύονద༻ɺόʔδϣϯΞοϓͳͲͷਓతίετΛेʹ͑ͳ͍ͳΒ ଟগෆศͰ(க໋తͰͳ͍ͳΒ)ϚωʔδυαʔϏεΛબ͠ ޙͷػೳ֦ுΛظͭͭ͠ӡ༻ͷखؒΛݮΒͨ͠΄͏͕Α͍
伱ؒՈ۩Λࣗ࡞͢Δ খ͘͞ɺͦͷϢʔεέʔεͰదͳ൚༻Λ࣋ͬͨͷΛ࡞Δ ຊՈ͕伱ؒΛຒΊͨΒࣺͯΒΕΔཻͰ࡞Δ
ʮ伱ؒՈ۩ OSSʯͷ࣮ྫͱઃܭࢥ
伱ؒͷ࣮ྫ - S3 ͱ Redshift ͷؒ Redshift ʹσʔλΛऔΓ͜Ήʹ S3 ͔Βͷίϐʔ͕ඞਢ
Redshift ʹଓ͠ɺऔΓࠐΉΦϒδΣΫτΛࢦఆͯ͠ COPY ΫΤϦΛൃߦ 2012-11 ʹ Redshift ͕ϩʔϯνͯ͠3 S3͔ΒܧଓతʹऔΓ͜ΉϚωʔδυͳํ๏ଘࡏ͠ͳ͔ͬͨ
伱ؒՈ۩ OSS ͷ࣮ྫ - Rin github.com/fujiwara/Rin S3 Πϕϯτ௨Ͱ SQS ʹૹ৴
SQS ͷϝοηʔδΛݩʹ Redshiftʹ COPY Λൃߦ ͯ͠औΓࠐΈΛߦ͏ Go πʔϧ 2015.05 ʹ։ൃ
ݩʑผͷํ๏Λ༻͍ͯͨ͠ fluent-plugin-redshift Fluentd ͷ output plugin ϝοηʔδͷoutputॲཧͱͯ͠ S3 Ξοϓϩʔυ Redshift
ͷCOPYൃߦ Λಉظతʹ࣮ߦ Fluentd ʹϩάΛૹΔ͚ͩͰ Redshift ·ͰऔΓ͜ ·ΕΔͷͰศརͱࢥͬͯಋೖ
ӡ༻͍ͯͯ͠ͷ S3ͷΞοϓϩʔυͱRedshiftͷऔΓࠐΈ͕ෆՄ Redshift ఆظతʹϝϯςφϯεͰఀࢭ ΫϥελϦαΠζͰఀࢭ ఀࢭ࣌ COPY ͕ࣦഊ͢Δ S3 ͷΞοϓϩʔυͱ
COPY ͕ҰମͷͨΊ ϦτϥΠΞοϓϩʔυ͔ΒΓͳ͓͠ → S3 ʹΰϛ͕ཷ·Δ
Rin ͷઃܭํ S3 ͷΞοϓϩʔυଞʹͤΔ (fluentd) S3, SQS ͷՄ༻ੑେมߴ͍ Redshift ͷμϯλΠϜൺֱతେ͖͍
ϛεϚον෦ΛҰʹॲཧ͠ͳ͍͜ͱͰ ϦτϥΠΛ༰қʹ Redshift ͷ COPY ൃߦʹಛԽ͢Δπʔϧ
Rin ͷઃఆϑΝΠϧ queue_name: my_queue_name # SQS queue name targets: -
s3: region: ap-northeast-1 bucket: my.test.bucket key_prefix: test/foo/ redshift: host: redshift.example.com port: 5439 dbname: test user: test password: xxxxxxxx schema: public table: foo sql_option: "JSON 'auto' GZIP"
ॊೈͳऔΓࠐΈઃఆ targets: - s3: bucket: my.test.bucket key_regexp: test/schema-([a-z0-9]+)/table-([a-z0-9]+)/ redshift: schema:
$1 # ^ͷਖ਼نදݱͰcaptureͨ͠Λల։ table: $2 େͷΞϓϦέʔγϣϯͰऔΓࠐΈςʔϒϧෳʹͳΔ ॊೈʹऔΓ͜ΊΔΑ͏ʹಈతઃఆΛՄೳʹ
։ൃͯ͠5ϲ݄ޙ 伱͕ؒຒ·ͬͨ 2015.10 Kinesis Firehose ൃද
Before / After Fluentd ϩάΛૹ৴ 㱺 Redshift ʹॱ࣍औΓ͜·ΕΔ ߏͦͷ··Ϛωʔδυʹ !
౦ژϦʔδϣϯʹདྷͨͷ 2017.07 (2ޙ)
Rin ͷण໋௵͑ͨͷ͔? ࣮·ͩੜ͖͍ͯ·͢ Fluentd͔ΒͷऔΓࠐΈFirehoseܦ༝ͰΑ͘ͳ͕ͬͨ ผͷϢʔεέʔεͰ͑Δ ELB / ALB ͷϩά͕ S3
ʹஔ͞ΕΔͷΛஞ࣍औΓࠐΈ͢Δͷʹ׆༂த ϢʔεέʔεΛߜͬͨదͳ൚༻ੑ͕ޭ͍ͯ͠Δ
Rin ͷڭ܇ Ͳ͏ߟ͑ͯϚωʔδυʹͳͬͯ΄͍͠ͷͦͷ͏ͪͳΔ (धཁ͕͋Ε…) ෳͷαʔϏεؒ࿈ܞΛҰʹॲཧ͢ΔͱϦτϥΠ͕ෳࡶʹͳΔ ୯ػೳʹಛԽͭͭ͠ɺͦͷυϝΠϯͰͷ൚༻ੑΛ࣋ͨͤΔͱ͍ճͤΔ Ϛωʔδυʹͳͬͨͱ͖ʹ͖Ε͍ʹऔΓ֎ͤΔઃܭ͕େࣄ
伱ؒՈ۩ OSS ͷྫ - s32cs Amazon CloudSearch 2012-04 ϦϦʔε 2019-10
ݱࡏ ϚωʔδυͰσʔλΛܧଓతʹऔΓ͜Ήํ๏ͳ͍ (console/ aws-cli / API / HTTP POST ͷΈ)
github.com/fujiwara/s32cs (S3 to CloudSearch) S3 ͷΠϕϯττϦΨͰىಈ͢Δ Lambda ؔͱ࣮ͯ͠ S3 ʹஔ͞Εͨ
ndjson (Newline Delimited JSON) ΛՃ HTTP POST Ͱ CloudSearch ʹೖ͢Δ
CloudSearch ͷೖॲཧ CloudSearch 5MB ҎԼͷ JSON ྻΛೖ ࡉ͔͍୯ҐͰೖ͢ΔͱύϑΥʔϚϯε͕ྼԽ͢Δ s32cs
= ྻܗࣜͷՃɺׂͱ HTTP POST Λͬͯ͘ΕΔ͚ͩͷπʔϧ {"id": "123", "type": "add", "fields": {"title": "hoge", "message": "Θ͍Θ͍"]}} {"id": "345", "type": "delete"} ↓ [ {"id": "123", "type": "add", "fields": {"title": "hoge", "message": "Θ͍Θ͍"]}}, {"id": "345", "type": "delete"} ]
Firehose → S3 ͷΠϕϯττϦΨ͔Βͳʹ͔͢Δύλʔϯ n͝ͱʹσʔλॲཧ͢ΔόονΛΠϕϯτυϦϒϯʹม͍ͯ͠Δ s32cs Ҏલ cronͰաڈ5ͷσʔλΛDB͔ΒऔಘɺՃɺHTTP POST Ͱೖ͢Δόον
σʔλऔಘɺՃɺೖ͕ҰମͰෆՄ s32cs Ҏޙ ੜΞϓϦέʔγϣϯͰͷϩάૹ৴ 5͝ͱʹσʔλΛ·ͱΊΔͷ Firehose ·ͱ·ͬͨσʔλΛՃͯ͠ೖ͢Δ͚͕ͩࣄ
ຊʹΓ͍ͨ͜ͱ = ॱ࣍ൃੜ͢ΔσʔλΛదͳ୯ҐͰॲཧ͍ͨ͠ ͜ΕΛn͝ͱͷcronͰ࣮ͯ͠͠·͏ͱ… 1ճॲཧ͕ൈ͚Δͱ࣍ճ࣮ߦͰ2ճΛॲཧ͢Δඞཁ͕ → ࠷ޙͷॲཧΛه͓ͯ͘͠ඞཁ͕͋Δ → γϦΞϧॲཧʹͳΔ →
Մ༻ੑύϑΥʔϚϯεͷͨΊͷࢄॲཧ͕ͮ͠Β͍ → ͠nΛॲཧ͠Α͏ͱ͢Δͱ…? εέʔϧ͠ͳ͍ + ࣦഊ͢Δͱۓு͕Δ
Firehose ͰͷετϦʔϜ → S3ΠϕϯττϦΨॲཧ n͝ͱͷcronΛସ ετϦʔϜΛn͝ͱͷchunkͰS3ʹు͖ग़͢ॲཧ = Firehose S3 ʹ
ࢦఆ࣌ؒ (60ʙ900ඵ) ͘͠ࢦఆαΠζ (1ʙ128MB)ͷ ઌʹୡͨ݅͠ͰΦϒδΣΫτ͕ੜ͞ΕΔ ࢄॲཧ͕ՄೳʹͳΔ όονॲཧଆʹঢ়ଶΛ࣋ͭඞཁ͕ͳ͍ ҆৺ͯ͠ΦϯϝϞϦॲཧ͕Ͱ͖Δʂ
ϚωʔδυαʔϏεʹ͍Ζ͍Ζ͋Δ ΫϥυωΠςΟϒ = ΠϯελϯεΛҙࣝ͠ͳͯ͘Α͍(Ͱ͖ͳ͍) SQS, S3, CloudWatch, Labmda, DynamoDB... ΫϥυωΠςΟϒʹ͍ۙ
= Πϯελϯεᐓؾʹݟ͑Δ͕εέʔϧ͕ࣗಈ ELB, CloudSearch... ΫϥυωΠςΟϒʹͳΓ͖Εͳ͍=Πϯελϯε͕ݟ͑εέʔϧ͕ࣗಈͰͳ͍ RDS, ElastiCache... ΫϥυωΠςΟϒͳͷΛ׆༻͍ͯ͘͠ͱεέʔϧ͍͢͠ ΫϥυωΠςΟϒͳࢥߟ๏େࣄ
s32cs ͷڭ܇ n͝ͱͷcronॲཧɺຊʹΓ͍ͨ͜ͱ࣌ؒґଘͰͳ͍͔ʁ దͳ୯ҐͰ੍͞ΕͨσʔλͷΠϕϯτυϦϒϯॲཧָ࣮͕ ؾ݉Ͷͳ͘ΦϯϝϞϦॲཧ͕Ͱ͖Δ ঢ়ଶϚωʔδυαʔϏεʹɺঢ়ଶΛ࣋ͨͳ͍ॲཧ͚ͩॻ͘ͱεέʔϧ͕༰қ
伱ؒՈ۩ OSS ͷྫ - ssmwrap github.com/handlename/ssmwrap SSM Parameter Store ͷΛڥมʹઃఆͯ͠ίϚϯυΛ
exec ͢Δ wrapper ੈͷதʹྨࣅ͍ͬͺ͍͋Δ͕͋͑ͯ… github.com/remind101/ssm-env github.com/okzk/env-injector github.com/jamietsao/aws-ssm-env
ssmwrap $ ssmwrap -path /prod/ -- mycommand ྫ͑ SSM ύϥϝʔλετΞʹ
/prod/DB_PASSWORD, /prod/API_TOKEN ͕ೖ͍ͬͯΔ߹ ڥม DB_PASSWORD, API_TOKEN ʹΛઃఆͯ͠ mycommand Λ࣮ߦ 2018.07 ECS task ʹ SSM ͷΛͨ͢Ίʹ։ൃ
ssmwrap ͷ͍͍ͱ͜Ζ exec ͢Δ(ࢠϓϩηεͱͯ͠ىಈ͠ͳ͍) wrapper ͳͷͰ ίϯςφͷ entrypoint ʹࢦఆͰ͖Δ #
ssmwrap -path /prod/ -- command ͱಉͷىಈॲཧ ENTRYPOINT ["/usr/bin/ssmwrap"] ENV SSMWRAP_PATHS=/prod/ CMD ["--", "mycommand"] ϦτϥΠ͕ࢦఆͰ͖Δ (ྫ -retries=3 or SSMWRAP_RETRIES=3) ύϥϝʔλετΞ͕ΤϥʔΛฦͨ͠߹ɺϦτϥΠͰ͖ͳ͍ͱ
͔͋Β͞·ͳ伱͙ؒ͢ຒ·Δ 2018-11 ECS task ʹ SSM ύϥϝʔλετΞͷΛڥมͱͯ͠ઃఆͰ͖Δ secrets ػೳϦϦʔε !
ssmwrap ͷར༻Ձͳ͘ͳͬͨͷ͔? ECS Ҏ֎Ͱ͑Δ EC2 Ͱಈ࡞͢Δ shell script ͷ shebang
ʹࢦఆͨ͠Γ #!/usr/bin/ssmwrap -path /prod/ -- /bin/sh echo $DB_PASS ... AWS֎ (ྫ͑ CircleCI) Ͱͷ࣮ߦʹಉ༷ʹࠐΊΔ γεςϜ͕ͯ͢ ECS Ͱͳ͍ঢ়ଶͰ·ͩ·ͩར༻Ձ͋Δ
ࢦఆͨ͠ΛϑΝΠϧͱͯ͠ॻ͖ग़͔ͯ͠Β exec ssmwrap -paths="/prod/" \ -file "Name=/prod/ID_RSA_KEY,Path=/root/.ssh/id_rsa,Mode=400" \ -file "Name=/prod/ID_RSA_PUB,Path=/root/.ssh/id_rsa.pub,Mode=644"
\ -- mycommand 伴ͳͲɺڥมͰͳ͘ϑΝΠϧஔ͕લఏͷΞϓϦέʔγϣϯଟ͍
GoͷϥΠϒϥϦͱͯ͑͠Δ ͜Ε͕໌ه͞Ε͍ͯΔྨࣅπʔϧগͳ͍(ͣ) godoc.org/github.com/handlename/ssmwrap#Export ϥΠϒϥϦͱͯ͑͠Δ → Lambda Ͱ͑Δ import "github.com/handlename/ssmwrap" err
:= ssmwrap.Export(ssmwrap.ExportOptions{ Paths: []string{"/prod/"}, Retries: 3, }) // ී௨ʹڥมΛࢀর͢Δ͚ͩ dbpass := os.Getenv("DB_PASS")
Lambda ͰύϥϝʔλετΞͷΛڥมʹ͍ͨ͠(Ͱ͢ΑͶ) ϚωʔδυͰઃఆ͢Δํ๏(·ͩ)ͳ͍͕Ͱ͖ͯવΔ͖ ϚωʔδυͰͰ͖ΔΑ͏ʹͳͬͨ߹ͷίʔυमਖ਼͕ۃগ ssmwrap.Export() Λআڈ͢Δ͚ͩ Lambda ͰύϥϝʔλετΞΛࢀর͢ΔίʔυΛॻ͘ΑΓݟ௨͕͠Α͍
ssmwrap ͷࢥ Ϛωʔδυʹͳͬͨͱ͖ʹऔΓ֎͍͢͠Α͏ͳઃܭ ڥม/ϑΝΠϧʹSSMύϥϝʔλετΞͷΛઃఆ͢Δ͜ͱʹूத͢Δ ECSɿsecrets ͰαϙʔτࡁΈ Lambda: ͦͷ͏ͪͰ͖Δ(͖ͬͱ) ϑΝΠϧʹॻ͖ग़͔ͯͯ͠͠Βίϯςφىಈ →
k8sͰͰ͖Δɺͭ·ΓECS… কདྷϚωʔδυʹͳΓͦ͏ͳ෦ΛΞϓϦέʔγϣϯͷͨॻ͖Ͱղܾ͠ͳ͍ খ͍͞πʔϧ/ϥΠϒϥϦʹΓग़͓ͯ͘͠ औΓ֎͕͠༰қʹͳΔ
ͦͷ͏ͪऔΓ֎͢ͷΛͳͥΘ͟Θ͟࡞Δͷ͔ ΞϓϦέʔγϣϯͰॻ͘ͱີ݁߹͢Δ औΓ֎͍ͨ࣌͠ʹີ݁߹͍ͯ͠Δͱվमͮ͠Β͍ ϥΠϒϥϦͰͳ͍ίʔυԟʑʹͯ͠ผϓϩδΣΫτʹίϐϖ͞ΕΔ ͍͟मਖ਼ΛೖΕΑ͏ͱͯ͠ίϐϖͳͷͰ… վमͮ͠Β͍ΞϓϦέʔγϣϯ → ͦͷ͏ͪ୭৮Γͨ͘ͳ͘ͳΔ վम͍͢͠ΞϓϦέʔγϣϯ →
ίετͰۙԽΛਐΊ͍͢
OSS ͱͯ͠࡞Δ
OSS ͱͯ͠࡞Δ ࣗΒ͔͠Θͳͯ͘ OSS ʹͯ͠͠·͏ ઃܭࢦͱͯ͠ ͜ΕΛ OSS ʹ͢ΔͳΒͲ͏͢Δ͔ Λߟ͑Δͷ༗༻
(ࣄͰ OSS ʹͰ͖ͳͯ͘)
OSS ͱͯ͠࡞Δཧ༝(1) υΩϡϝϯτΛॻ͘ؾʹͳΔ README ͙Β͍ؤுͬͯॻ͘ϞνϕʔγϣϯʹͳΔ ۭͩͱஏ͔͍ͣ͠ͷͰ……
OSS ͱͯ͠࡞Δཧ༝(2) աͳࣾࣄͷࠞೖΛ͙ ʮࣾ/ͦͷϓϩδΣΫτҎ֎ʹҙຯ͕͋Δػೳͳͷ͔?ʯ քΛݟۃΊΔ͜ͱͰকདྷ֎͘͢͠ͳΔ ઃܭ͕͖Ε͍ʹͳΔ
OSS ͱͯ͠࡞Δཧ༝(3) ຐվ൛͕૿৩͢ΔͷΛ͙ ʮGitHubͰόΠφϦఏڙͯ͠ΔͷͰͦΕΛ͍ͬͯͩ͘͞ʯ ʮRubyGems / CPAN / npm /
etc. ͔ΒΠϯετʔϧ͍ͯͩ͘͠͞ʯ ಠཱͨ͠ύοέʔδʹͳͬͯͳ͍ͱίϐϖ͞ΕΔ ίϐϖޙʹվ͞ΕΔͱ৽ػೳόάϑΟοΫεʹैͰ͖ͳ͍
OSS ͱͯ͠࡞Δཧ༝(4) ಉ͡Α͏ͳ伱ؒՈ۩ΛΈΜͳ͕ຖճखॻ͖͢Δͷແବʂʂ ͱ͍͑ϢʔεέʔεʹΑͬͯ࠶࣮͢Δ͜ͱ (ssmwrap)
·ͱΊ AWSͷϚωʔδυαʔϏεʹʮ伱ؒʯ͕͋Δ 伱ؒՈ۩Λ࡞ͬͯΑΓΑ͍ӡ༻Λ ඞཁʹԠͯ͡औΓ֎ͤΔΑ͏ʹઃܭ͢Δ OSS ͱͯ͠࡞Δ͜ͱΛߟ͑Δ͜ͱͰɺΑΓΑ͍ઃܭͱ࣮ʹ ཁͪΌΜͱ͑·͠ΐ͏ʂ
Thank You!