日々大量の素材を扱うために改善したこと/webgenba-meetup-03

2018/03/07 Web現場Meetup #3

1. ೔ʑେྔͷૉࡐΛѻ͏ͨΊ ʹվળͨ͜͠ͱ 2018/03/07 Webݱ৔Meetup ϐΫελגࣜձࣾɹҏ౻ ༟ر mits / 33114786
2. ࣗݾ঺հ • ҏ౻ ༟ر • 2015/08 ʙ ϐΫελגࣜձࣾ • 2017/01 ʙ PIXTA VIETNAM ʹग़޲ • 2018/01 ʙ ٕज़ਪਐάϧʔϓ 
3. ϐΫελגࣜձࣾͷ͝঺հ 
4. ձࣾ֓ཁ • ઃཱ೔: 2005/08/25 • ཧ೦: ࠽ೳΛͭͳ͗ɺੈքΛϙδςΟϒʹ͢Δ • ࣄۀ಺༰: PIXTA, fotowa, Snapmart ͷӡӦ • ڌ఺: ೔ຊ, ୆࿷, λΠ, γϯΨϙʔϧ, ϕτφϜ, ؖࠃ 
5. https://pixta.jp ೔ຊޠΛؚΉ6ݴޠͰల։த 
6. ࠓ೔࿩͢͜ͱ • എܠ • ࣮ࢪ಺༰ • ࠓޙͷ՝୊ • ·ͱΊ 
7. എܠ 
8. ૉࡐ͕ PIXTA Ͱൢച͞ΕΔ·Ͱ 1. Ξοϓϩʔυ 2. ਃ੥ʢૉࡐʹλΠτϧɾλά౳Λඥ෇͚ʣ 3. ৹ࠪʢૉࡐͷΫΦϦςΟ౳Λ൑அʣ 4. ൢച։࢝ʂ 
9. ૉࡐ͕ PIXTA Ͱൢച͞ΕΔ·Ͱ 1. Ξοϓϩʔυ 2. ਃ੥ʢૉࡐʹλΠτϧɾλά౳Λඥ෇͚ʣ 3. ৹ࠪʢૉࡐͷΫΦϦςΟ౳Λ൑அʣ 4. ൢച։࢝ʂ 
10. Ξοϓϩʔυ • ϚΠϖʔδܦ༝ͷΞοϓϩʔυ • গྔͷૉࡐΛ PIXTA Ͱൢച͍ͨ͠ࡍʹ࢖༻ • FTPαʔόʔܦ༝ͷΞοϓϩʔυ • େྔͷૉࡐΛ PIXTA Ͱൢച͍ͨ͠ࡍʹ࢖༻ 
11. Ξοϓϩʔυ • ϚΠϖʔδܦ༝ͷΞοϓϩʔυ • গྔͷૉࡐΛ PIXTA Ͱൢച͍ͨ͠ࡍʹ࢖༻ • FTPαʔόʔܦ༝ͷΞοϓϩʔυ • େྔͷૉࡐΛ PIXTA Ͱൢച͍ͨ͠ࡍʹ࢖༻ 
12. FTPαʔόʔܦ༝ͷΞοϓϩʔυ • ετοΫϑΥτۀքͰ͸Ұൠతͳํ๏ • େྔͷૉࡐΛอ͍࣋ͯ͠Δ ΫϦΤΠλʔ ͕ޮ ཰Α͘ΞοϓϩʔυͰ͖ΔͨΊʹଘࡏ͢Δ  FTP Server ΫϦΤΠλʔ Ξοϓϩʔυ
13. چFTPΞοϓϩʔυγεςϜ  PIXTA (Batch) S3 FTP Server ΫϦΤΠλʔ ᶃΞοϓϩʔυ PIXTA (Admin) ϐΫελ ελοϑ ᶄ࿈བྷʢϚΠϖʔδ্ʣ ᶅFTP=>S3ಉظґཔ ᶅFTP=>S3ಉظґཔ ᶆS3ಉظ ᶇ݁ՌΛ௨஌ ᶈαϜωΠϧੜ੒ ᶉૉࡐ৘ใΛอଘ Worker
14. چFTPΞοϓϩʔυγεςϜ  PIXTA (Batch) S3 FTP Server ΫϦΤΠλʔ ᶃΞοϓϩʔυ PIXTA (Admin) ϐΫελ ελοϑ ᶄ࿈བྷʢϚΠϖʔδ্ʣ ᶅFTP=>S3ಉظґཔ ᶅFTP=>S3ಉظґཔ ᶆS3ಉظ ᶇ݁ՌΛ௨஌ ᶈαϜωΠϧੜ੒ ᶉૉࡐ৘ใΛอଘ Worker
15. Ξοϓϩʔυ݅਺ͷਪҠ  ,  ,  ,  ,  , ೥ ೥ ೥ ೥ ೥ ೥
16. چFTPΞοϓϩʔυγεςϜ  PIXTA (Batch) S3 FTP Server ΫϦΤΠλʔ ᶃΞοϓϩʔυ PIXTA (Admin) ϐΫελ ελοϑ ᶄ࿈བྷʢϚΠϖʔδ্ʣ ᶅFTP=>S3ಉظґཔ ᶅFTP=>S3ಉظґཔ ᶆS3ಉظ ᶇ݁ՌΛ௨஌ ᶈαϜωΠϧੜ੒ ᶉૉࡐ৘ใΛอଘ Worker
17. ໰୊఺ • ॲཧϑϩʔͷෳࡶ͞ʹΑΔӨڹ • Ξοϓϩʔυ݅਺૿Ճ܏޲ʹΑΔɺ͞ΒͳΔ γεςϜͷεέʔϧԽ 
18. ࣮ࢪ಺༰ 
19. ໰୊఺ • ॲཧϑϩʔͷෳࡶ͞ʹΑΔӨڹ • Ξοϓϩʔυ݅਺૿Ճ܏޲ʹΑΔɺ͞ΒͳΔ γεςϜͷεέʔϧԽ 
20. ৽FTPΞοϓϩʔυγεςϜ  PIXTA (API) S3 FTP Server Lambda ΫϦΤΠλʔ ᶃΞοϓϩʔυ ᶄS3ʹಉظ ᶅΠϕϯττϦΨ ᶆαϜωΠϧੜ੒ʢಈըʣ ᶆαϜωΠϧอଘʢը૾ʣ ᶆૉࡐ৘ใΛอଘ S3 Worker
21. ໰୊఺ • ॲཧϑϩʔͷෳࡶ͞ʹΑΔӨڹ • Ξοϓϩʔυ݅਺૿Ճ܏޲ʹΑΔɺ͞ΒͳΔ γεςϜͷεέʔϧԽ 
22. چFTPΞοϓϩʔυγεςϜ  PIXTA (Batch) S3 FTP Server ΫϦΤΠλʔ ᶃΞοϓϩʔυ PIXTA (Admin) ϐΫελ ελοϑ ᶄ࿈བྷʢϚΠϖʔδ্ʣ ᶅFTP=>S3ಉظґཔ ᶅFTP=>S3ಉظґཔ ᶆS3ಉظ ᶇ݁ՌΛ௨஌ ᶈαϜωΠϧੜ੒ ᶉૉࡐ৘ใΛอଘ Worker
23. ৽FTPΞοϓϩʔυγεςϜ  PIXTA (API) S3 FTP Server Lambda ΫϦΤΠλʔ ᶃΞοϓϩʔυ ᶄS3ʹಉظ ᶅΠϕϯττϦΨ ᶆαϜωΠϧੜ੒ʢಈըʣ ᶆαϜωΠϧอଘʢը૾ʣ ᶆૉࡐ৘ใΛอଘ S3 Worker
24. ໰୊఺ • ॲཧϑϩʔͷෳࡶ͞ʹΑΔӨڹ • Ξοϓϩʔυ݅਺૿Ճ܏޲ʹΑΔɺ͞ΒͳΔ γεςϜͷεέʔϧԽ 
25. چFTPΞοϓϩʔυγεςϜ  PIXTA (Batch) S3 FTP Server ΫϦΤΠλʔ ᶃΞοϓϩʔυ PIXTA (Admin) ϐΫελ ελοϑ ᶄ࿈བྷʢϚΠϖʔδ্ʣ ᶅFTP=>S3ಉظґཔ ᶅFTP=>S3ಉظґཔ ᶆS3ಉظ ᶇ݁ՌΛ௨஌ ᶈαϜωΠϧੜ੒ ᶉૉࡐ৘ใΛอଘ Worker
26. ৽FTPΞοϓϩʔυγεςϜ  PIXTA (API) S3 FTP Server Lambda ΫϦΤΠλʔ ᶃΞοϓϩʔυ ᶄS3ʹಉظ ᶅΠϕϯττϦΨ ᶆαϜωΠϧੜ੒ʢಈըʣ ᶆαϜωΠϧอଘʢը૾ʣ ᶆૉࡐ৘ใΛอଘ S3 Worker
27. AWS Lambda ͱ͸ 
28. ৽FTPΞοϓϩʔυγεςϜ  PIXTA (API) S3 FTP Server Lambda ΫϦΤΠλʔ ᶃΞοϓϩʔυ ᶄS3ʹಉظ ᶅΠϕϯττϦΨ ᶆαϜωΠϧੜ੒ʢಈըʣ ᶆαϜωΠϧอଘʢը૾ʣ ᶆૉࡐ৘ใΛอଘ S3 Worker
29. • Inotify + Ruby ͰϑΝΠϧγεςϜΛ؂ࢹ • S3ʹϑΝΠϧΛಉظ ᶄS3ʹಉظ 
30. ৽FTPΞοϓϩʔυγεςϜ  PIXTA (API) S3 FTP Server Lambda ΫϦΤΠλʔ ᶃΞοϓϩʔυ ᶄS3ʹಉظ ᶅΠϕϯττϦΨ ᶆαϜωΠϧੜ੒ʢಈըʣ ᶆαϜωΠϧอଘʢը૾ʣ ᶆૉࡐ৘ใΛอଘ S3 Worker
31. ᶅΠϕϯττϦΨ • ΠϕϯττϦΨʹ S3 Λ௥Ճ • Suffix ʹର৅ͱͳΔϑΝΠϧΛࢦఆ • ը૾ɺಈըͦΕͧΕͷ֦ுࢠ • Event types ʹର৅ͱͳΔΠϕϯτΛࢦఆ • ObjectCreatedͰΦϒδΣΫτ࡞੒Λର৅ 
32. ৽FTPΞοϓϩʔυγεςϜ  PIXTA (API) S3 FTP Server Lambda ΫϦΤΠλʔ ᶃΞοϓϩʔυ ᶄS3ʹಉظ ᶅΠϕϯττϦΨ ᶆαϜωΠϧੜ੒ʢಈըʣ ᶆαϜωΠϧอଘʢը૾ʣ ᶆૉࡐ৘ใΛอଘ S3 Worker
33. ᶆLambda • ը૾ϑΝΠϧʹ͍ͭͯ͸ Lambda ಺ͰαϜω ΠϧΛੜ੒ͯ͠ɺS3 ʹอଘ • ಈըϑΝΠϧʹ͍ͭͯ͸ αϜωΠϧੜ੒Λ SQSܦ༝Ͱґཔ • ૉࡐ৘ใΛ APIܦ༝Ͱอଘ 
34. ᶆLambda(ը૾αϜωΠϧੜ੒) import GraphicsMagick from 'gm'; export default class JpgProcessor { constructor({ awsRegion, srcBucket, srcKey }) { . this.gm = GraphicsMagick.subClass({ imageMagick: true }); . } processImage() { return this._resizeThumbnail(); } _resizeThumbnail() { return this.gm(this.imageDataBody) .resize(THUMBNAIL_SIZE, THUMBNAIL_SIZE) .toBuffer(this.imageType, (err, buffer) => { . . }; } }
35. ݁Ռ • ॲཧϑϩʔͷෳࡶ͞ʹΑΔӨڹ • ૉࡐΞοϓϩʔυΛτϦΨʹ͢Δ͜ͱͰɺॲཧϑϩʔ ΛҰํํ޲ʹͯ͠ෳࡶ͞Λͳͨ͘͠ • Ξοϓϩʔυ݅਺૿Ճ܏޲ʹΑΔɺ͞ΒͳΔγεςϜͷ εέʔϧԽ • ϐΫελελοϑ΁ͷ࿈བྷΛͳ͘͠ɺLambdaΛ࢖༻ ͢Δ͜ͱͰεέʔϧԽΛਤͬͨ 
36. ࠓޙͷ՝୊ 
37. Amazon EBS ͷϚ΢ϯτ • ౰ॳɺFTPαʔόʔ্Ͱ s3fs Λ࢖ͬͯ s3 ΛϚ΢ ϯτ͍͕ͯͨ͠ɺύϑΥʔϚϯε͕ग़ͳ͔ͬͨ • FTPαʔόʔ Ͱ͸ɺ1000GBͷEBSΛϚ΢ϯτ ͍ͯ͠Δ • 5୆ಈ͍͍ͯͯίετ͕ 500$ ~ 1000$ 
38. Amazon EBS ͷϚ΢ϯτ 
39. ·ͱΊ 
40. ·ͱΊ • ॲཧϑϩʔͷෳࡶ͞ʹΑΔӨڹ • ૉࡐΞοϓϩʔυΛτϦΨʹ͢Δ͜ͱͰɺॲཧϑϩʔ ΛҰํํ޲ʹͯ͠ෳࡶ͞Λͳͨ͘͠ • Ξοϓϩʔυ݅਺૿Ճ܏޲ʹΑΔɺ͞ΒͳΔγεςϜͷ εέʔϧԽ • ϐΫελελοϑ΁ͷ࿈བྷΛͳ͘͠ɺLambdaΛ࢖༻ ͢Δ͜ͱͰεέʔϧԽΛਤͬͨ 
41. We are hiring! 
42. ͝ਗ਼ௌ͋Γ͕ͱ͏͍͟͝·ͨ͠ʂ  mits / 33114786