Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
日々大量の素材を扱うために改善したこと/webgenba-meetup-03
Search
hamuyuuki
March 08, 2018
Technology
0
1.1k
日々大量の素材を扱うために改善したこと/webgenba-meetup-03
2018/03/07 Web現場Meetup #3
hamuyuuki
March 08, 2018
Tweet
Share
More Decks by hamuyuuki
See All by hamuyuuki
VS CodeのExtensionを作ってみた/pixta-techcamp-05
hamuyuuki
0
1.3k
Other Decks in Technology
See All in Technology
Tokyo_reInforce_2025_recap_iam_access_analyzer
hiashisan
0
130
GeminiとNotebookLMによる金融実務の業務革新
abenben
0
240
開発生産性を組織全体の「生産性」へ! 部門間連携の壁を越える実践的ステップ
sudo5in5k
0
150
Delegating the chores of authenticating users to Keycloak
ahus1
0
130
製造業からパッケージ製品まで、あらゆる領域をカバー!生成AIを利用したテストシナリオ生成 / 20250627 Suguru Ishii
shift_evolve
PRO
1
160
Witchcraft for Memory
pocke
1
640
解析の定理証明実践@Lean 4
dec9ue
1
190
How Community Opened Global Doors
hiroramos4
PRO
1
130
論文紹介:LLMDet (CVPR2025 Highlight)
tattaka
0
170
プロダクトエンジニアリング組織への歩み、その現在地 / Our journey to becoming a product engineering organization
hiro_torii
0
130
ひとり情シスなCTOがLLMと始めるオペレーション最適化 / CTO's LLM-Powered Ops
yamitzky
0
460
【5分でわかる】セーフィー エンジニア向け会社紹介
safie_recruit
0
26k
Featured
See All Featured
The Language of Interfaces
destraynor
158
25k
Music & Morning Musume
bryan
46
6.6k
How to Think Like a Performance Engineer
csswizardry
24
1.7k
The Web Performance Landscape in 2024 [PerfNow 2024]
tammyeverts
8
680
Performance Is Good for Brains [We Love Speed 2024]
tammyeverts
10
940
How STYLIGHT went responsive
nonsquared
100
5.6k
Fantastic passwords and where to find them - at NoRuKo
philnash
51
3.3k
Writing Fast Ruby
sferik
628
62k
Making Projects Easy
brettharned
116
6.3k
Adopting Sorbet at Scale
ufuk
77
9.4k
Practical Orchestrator
shlominoach
188
11k
Done Done
chrislema
184
16k
Transcript
ʑେྔͷૉࡐΛѻ͏ͨΊ ʹվળͨ͜͠ͱ 2018/03/07 WebݱMeetup ϐΫελגࣜձࣾɹҏ౻ ༟ر mits / 33114786
ࣗݾհ • ҏ౻ ༟ر • 2015/08 ʙ ϐΫελגࣜձࣾ • 2017/01
ʙ PIXTA VIETNAM ʹग़ • 2018/01 ʙ ٕज़ਪਐάϧʔϓ
ϐΫελגࣜձࣾͷ͝հ
ձࣾ֓ཁ • ઃཱ: 2005/08/25 • ཧ೦: ࠽ೳΛͭͳ͗ɺੈքΛϙδςΟϒʹ͢Δ • ࣄۀ༰: PIXTA,
fotowa, Snapmart ͷӡӦ • ڌ: ຊ, , λΠ, γϯΨϙʔϧ, ϕτφϜ, ؖࠃ
https://pixta.jp ຊޠΛؚΉ6ݴޠͰల։த
ࠓ͢͜ͱ • എܠ • ࣮ࢪ༰ • ࠓޙͷ՝ • ·ͱΊ
എܠ
ૉࡐ͕ PIXTA Ͱൢച͞ΕΔ·Ͱ 1. Ξοϓϩʔυ 2. ਃʢૉࡐʹλΠτϧɾλάΛඥ͚ʣ 3. ৹ࠪʢૉࡐͷΫΦϦςΟΛஅʣ 4.
ൢച։࢝ʂ
ૉࡐ͕ PIXTA Ͱൢച͞ΕΔ·Ͱ 1. Ξοϓϩʔυ 2. ਃʢૉࡐʹλΠτϧɾλάΛඥ͚ʣ 3. ৹ࠪʢૉࡐͷΫΦϦςΟΛஅʣ 4.
ൢച։࢝ʂ
Ξοϓϩʔυ • ϚΠϖʔδܦ༝ͷΞοϓϩʔυ • গྔͷૉࡐΛ PIXTA Ͱൢച͍ͨ͠ࡍʹ༻ • FTPαʔόʔܦ༝ͷΞοϓϩʔυ •
େྔͷૉࡐΛ PIXTA Ͱൢച͍ͨ͠ࡍʹ༻
Ξοϓϩʔυ • ϚΠϖʔδܦ༝ͷΞοϓϩʔυ • গྔͷૉࡐΛ PIXTA Ͱൢച͍ͨ͠ࡍʹ༻ • FTPαʔόʔܦ༝ͷΞοϓϩʔυ •
େྔͷૉࡐΛ PIXTA Ͱൢച͍ͨ͠ࡍʹ༻
FTPαʔόʔܦ༝ͷΞοϓϩʔυ • ετοΫϑΥτۀքͰҰൠతͳํ๏ • େྔͷૉࡐΛอ͍࣋ͯ͠Δ ΫϦΤΠλʔ ͕ޮ Α͘ΞοϓϩʔυͰ͖ΔͨΊʹଘࡏ͢Δ FTP
Server ΫϦΤΠλʔ Ξοϓϩʔυ
چFTPΞοϓϩʔυγεςϜ PIXTA (Batch) S3 FTP Server ΫϦΤΠλʔ ᶃΞοϓϩʔυ PIXTA
(Admin) ϐΫελ ελοϑ ᶄ࿈བྷʢϚΠϖʔδ্ʣ ᶅFTP=>S3ಉظґཔ ᶅFTP=>S3ಉظґཔ ᶆS3ಉظ ᶇ݁ՌΛ௨ ᶈαϜωΠϧੜ ᶉૉࡐใΛอଘ Worker
چFTPΞοϓϩʔυγεςϜ PIXTA (Batch) S3 FTP Server ΫϦΤΠλʔ ᶃΞοϓϩʔυ PIXTA
(Admin) ϐΫελ ελοϑ ᶄ࿈བྷʢϚΠϖʔδ্ʣ ᶅFTP=>S3ಉظґཔ ᶅFTP=>S3ಉظґཔ ᶆS3ಉظ ᶇ݁ՌΛ௨ ᶈαϜωΠϧੜ ᶉૉࡐใΛอଘ Worker
Ξοϓϩʔυ݅ͷਪҠ , , , ,
,
چFTPΞοϓϩʔυγεςϜ PIXTA (Batch) S3 FTP Server ΫϦΤΠλʔ ᶃΞοϓϩʔυ PIXTA
(Admin) ϐΫελ ελοϑ ᶄ࿈བྷʢϚΠϖʔδ্ʣ ᶅFTP=>S3ಉظґཔ ᶅFTP=>S3ಉظґཔ ᶆS3ಉظ ᶇ݁ՌΛ௨ ᶈαϜωΠϧੜ ᶉૉࡐใΛอଘ Worker
• ॲཧϑϩʔͷෳࡶ͞ʹΑΔӨڹ • Ξοϓϩʔυ݅૿ՃʹΑΔɺ͞ΒͳΔ γεςϜͷεέʔϧԽ
࣮ࢪ༰
• ॲཧϑϩʔͷෳࡶ͞ʹΑΔӨڹ • Ξοϓϩʔυ݅૿ՃʹΑΔɺ͞ΒͳΔ γεςϜͷεέʔϧԽ
৽FTPΞοϓϩʔυγεςϜ PIXTA (API) S3 FTP Server Lambda ΫϦΤΠλʔ ᶃΞοϓϩʔυ
ᶄS3ʹಉظ ᶅΠϕϯττϦΨ ᶆαϜωΠϧੜʢಈըʣ ᶆαϜωΠϧอଘʢը૾ʣ ᶆૉࡐใΛอଘ S3 Worker
• ॲཧϑϩʔͷෳࡶ͞ʹΑΔӨڹ • Ξοϓϩʔυ݅૿ՃʹΑΔɺ͞ΒͳΔ γεςϜͷεέʔϧԽ
چFTPΞοϓϩʔυγεςϜ PIXTA (Batch) S3 FTP Server ΫϦΤΠλʔ ᶃΞοϓϩʔυ PIXTA
(Admin) ϐΫελ ελοϑ ᶄ࿈བྷʢϚΠϖʔδ্ʣ ᶅFTP=>S3ಉظґཔ ᶅFTP=>S3ಉظґཔ ᶆS3ಉظ ᶇ݁ՌΛ௨ ᶈαϜωΠϧੜ ᶉૉࡐใΛอଘ Worker
৽FTPΞοϓϩʔυγεςϜ PIXTA (API) S3 FTP Server Lambda ΫϦΤΠλʔ ᶃΞοϓϩʔυ
ᶄS3ʹಉظ ᶅΠϕϯττϦΨ ᶆαϜωΠϧੜʢಈըʣ ᶆαϜωΠϧอଘʢը૾ʣ ᶆૉࡐใΛอଘ S3 Worker
• ॲཧϑϩʔͷෳࡶ͞ʹΑΔӨڹ • Ξοϓϩʔυ݅૿ՃʹΑΔɺ͞ΒͳΔ γεςϜͷεέʔϧԽ
چFTPΞοϓϩʔυγεςϜ PIXTA (Batch) S3 FTP Server ΫϦΤΠλʔ ᶃΞοϓϩʔυ PIXTA
(Admin) ϐΫελ ελοϑ ᶄ࿈བྷʢϚΠϖʔδ্ʣ ᶅFTP=>S3ಉظґཔ ᶅFTP=>S3ಉظґཔ ᶆS3ಉظ ᶇ݁ՌΛ௨ ᶈαϜωΠϧੜ ᶉૉࡐใΛอଘ Worker
৽FTPΞοϓϩʔυγεςϜ PIXTA (API) S3 FTP Server Lambda ΫϦΤΠλʔ ᶃΞοϓϩʔυ
ᶄS3ʹಉظ ᶅΠϕϯττϦΨ ᶆαϜωΠϧੜʢಈըʣ ᶆαϜωΠϧอଘʢը૾ʣ ᶆૉࡐใΛอଘ S3 Worker
AWS Lambda ͱ
৽FTPΞοϓϩʔυγεςϜ PIXTA (API) S3 FTP Server Lambda ΫϦΤΠλʔ ᶃΞοϓϩʔυ
ᶄS3ʹಉظ ᶅΠϕϯττϦΨ ᶆαϜωΠϧੜʢಈըʣ ᶆαϜωΠϧอଘʢը૾ʣ ᶆૉࡐใΛอଘ S3 Worker
• Inotify + Ruby ͰϑΝΠϧγεςϜΛࢹ • S3ʹϑΝΠϧΛಉظ ᶄS3ʹಉظ
৽FTPΞοϓϩʔυγεςϜ PIXTA (API) S3 FTP Server Lambda ΫϦΤΠλʔ ᶃΞοϓϩʔυ
ᶄS3ʹಉظ ᶅΠϕϯττϦΨ ᶆαϜωΠϧੜʢಈըʣ ᶆαϜωΠϧอଘʢը૾ʣ ᶆૉࡐใΛอଘ S3 Worker
ᶅΠϕϯττϦΨ • ΠϕϯττϦΨʹ S3 ΛՃ • Suffix ʹରͱͳΔϑΝΠϧΛࢦఆ • ը૾ɺಈըͦΕͧΕͷ֦ுࢠ
• Event types ʹରͱͳΔΠϕϯτΛࢦఆ • ObjectCreatedͰΦϒδΣΫτ࡞Λର
৽FTPΞοϓϩʔυγεςϜ PIXTA (API) S3 FTP Server Lambda ΫϦΤΠλʔ ᶃΞοϓϩʔυ
ᶄS3ʹಉظ ᶅΠϕϯττϦΨ ᶆαϜωΠϧੜʢಈըʣ ᶆαϜωΠϧอଘʢը૾ʣ ᶆૉࡐใΛอଘ S3 Worker
ᶆLambda • ը૾ϑΝΠϧʹ͍ͭͯ Lambda ͰαϜω ΠϧΛੜͯ͠ɺS3 ʹอଘ • ಈըϑΝΠϧʹ͍ͭͯ αϜωΠϧੜΛ
SQSܦ༝Ͱґཔ • ૉࡐใΛ APIܦ༝Ͱอଘ
ᶆLambda(ը૾αϜωΠϧੜ) import GraphicsMagick from 'gm'; export default class JpgProcessor {
constructor({ awsRegion, srcBucket, srcKey }) { . this.gm = GraphicsMagick.subClass({ imageMagick: true }); . } processImage() { return this._resizeThumbnail(); } _resizeThumbnail() { return this.gm(this.imageDataBody) .resize(THUMBNAIL_SIZE, THUMBNAIL_SIZE) .toBuffer(this.imageType, (err, buffer) => { . . }; } }
݁Ռ • ॲཧϑϩʔͷෳࡶ͞ʹΑΔӨڹ • ૉࡐΞοϓϩʔυΛτϦΨʹ͢Δ͜ͱͰɺॲཧϑϩʔ ΛҰํํʹͯ͠ෳࡶ͞Λͳͨ͘͠ • Ξοϓϩʔυ݅૿ՃʹΑΔɺ͞ΒͳΔγεςϜͷ εέʔϧԽ •
ϐΫελελοϑͷ࿈བྷΛͳ͘͠ɺLambdaΛ༻ ͢Δ͜ͱͰεέʔϧԽΛਤͬͨ
ࠓޙͷ՝
Amazon EBS ͷϚϯτ • ॳɺFTPαʔόʔ্Ͱ s3fs Λͬͯ s3 ΛϚ ϯτ͍͕ͯͨ͠ɺύϑΥʔϚϯε͕ग़ͳ͔ͬͨ
• FTPαʔόʔ Ͱɺ1000GBͷEBSΛϚϯτ ͍ͯ͠Δ • 5ಈ͍͍ͯͯίετ͕ 500$ ~ 1000$
Amazon EBS ͷϚϯτ
·ͱΊ
·ͱΊ • ॲཧϑϩʔͷෳࡶ͞ʹΑΔӨڹ • ૉࡐΞοϓϩʔυΛτϦΨʹ͢Δ͜ͱͰɺॲཧϑϩʔ ΛҰํํʹͯ͠ෳࡶ͞Λͳͨ͘͠ • Ξοϓϩʔυ݅૿ՃʹΑΔɺ͞ΒͳΔγεςϜͷ εέʔϧԽ •
ϐΫελελοϑͷ࿈བྷΛͳ͘͠ɺLambdaΛ༻ ͢Δ͜ͱͰεέʔϧԽΛਤͬͨ
We are hiring!
͝ਗ਼ௌ͋Γ͕ͱ͏͍͟͝·ͨ͠ʂ mits / 33114786