create own CMS from scratch

by sota ohara

Slide 1

Slide 1 text

Slide 2

Slide 2 text

2 Creating Serverless CMS from Scratch Sota Ohara JS Fest @ Kyiv November 8, 2019

Slide 3

Slide 3 text

4 ota Ohara  sottar_  sottar  Frontend Engineer, ech Lead  at Mercari 

Slide 4

Slide 4 text

5 What Is Mercari?

Slide 5

Slide 5 text

6 About February 1st, 2013 Established Tokyo, Sendai, Fukuoka, Palo Alto, Portland, Boston Ofﬁces 1800+ Including subsidiaries Headcount 6

Slide 6

Slide 6 text

7 By the Numbers (JP/Full Year) 232 346.8 GMV¹ 530.2 B  In: billion JPY FY 06/2017 FY 06/2018 21.2 33.4 Net Sales 51.6 B  In: billion JPY FY 06/2017 FY 06/2018 8.45 10.75 MAU² 14.50 M  In: Million people FY 06/2017 FY 06/2018 ource: Internal documents, from FY2018.6 Presentation Material  1. GM after cancellation  2. Monthly Active sers in June. Number of registered users that used our app in the month  490.2 FY 06/2019 46.2 FY 06/2019 13.57 FY 06/2019

Slide 7

Slide 7 text

8 Global Non-Japanese in the company 20% 40% Non-Japanese engineers 40 nationalities represented in the workforce

Slide 8

Slide 8 text

9 Campaign Pages

Slide 9

Slide 9 text

10 Campaign Pages provide 5% off ticket for customer participate button for this campaign deadline of this campaign

Slide 10

Slide 10 text

11 Campaign Pages giving 500 yen (max) points if customer use the new feature use new feature button

Slide 11

Slide 11 text

12 Campaign Pages free commission fee campaign

Slide 12

Slide 12 text

13 Campaign Pages

Slide 13

Slide 13 text

14 Before Building the CMS PM Designer Frontend Engineer Decide on the target, specs, content... Ask design to create Create design using ﬁgma, sketch...

Slide 14

Slide 14 text

15 Before Building the CMS PM Frontend Engineer Conﬁrm design Write pug, css with the design Ask to create Ask to create (if jQuery needed) Write jQuery Designer

Slide 15

Slide 15 text

16 Ask for QA Ask for code review PM Frontend Engineer Review code Complete development Before Building the CMS QA check behaivior Designer

Slide 16

Slide 16 text

17 - Designer writes the code (pug & css) - Frontend Engineer reviews the code - Frontend Engineer writes JavaScript if needed - QA checks behavior in the staging environment - If we want to update content - even one word - we need to repeat the process again Before Building the CMS

Slide 17

Slide 17 text

18 - Designer writes the code (pug & css) - Frontend Engineer reviews the code - Frontend Engineer writes JavaScript if needed - QA checks behavior in the staging environment - If we want to update content - even one word - we need to repeat the process again Designers need to acquire knowledge about pug & css… Frontend Engineers need to take time to review code... Before Building the CMS

Slide 18

Slide 18 text

19 - Designer writes the code (pug & css) - Frontend Engineer reviews the code - Frontend Engineer writes JavaScript if needed - QA checks behavior in the staging environment - If we want to update content - even one word - we need to repeat the process again We have to write a jQuery if JavaScript code is required (api request, dynamic movement…) Before Building the CMS

Slide 19

Slide 19 text

20 - Designer writes the code (pug & css) - Frontend Engineer reviews the code - Frontend Engineer writes JavaScript if needed - QA checks behavior in the staging environment - If we want to update content - even one word - we need to repeat the process again Before Building the CMS

Slide 20

Slide 20 text

21 => Takes time to create a campaign page => Not designed for long-term operation Hard to operate many campaigns using this schema Before Building the CMS

Slide 21

Slide 21 text

22 Why We Created it from Scratch

Slide 22

Slide 22 text

23 What We Want to Create

Slide 23

Slide 23 text

24 What We Want to Create

Slide 24

Slide 24 text

25 What We Want to Create

Slide 25

Slide 25 text

26 Image Title Button Q&A Text LinkText What We Want to Create

Slide 26

Slide 26 text

27 We decided on features such as… - There are two pages - listing page and editor page - User can create the new campaign page via listing page - On editor page, user can combine components to create pages What We Want to Create

Slide 27

Slide 27 text

28 Pros - Reduce time needed to create a page - Designers are not required to have programming skill - Easy to update content What We Want to Create

Slide 28

Slide 28 text

29 Cons - Need to implement a component if the campaign page requires a new component which isn’t provided => Will be decreasing What We Want to Create

Slide 29

Slide 29 text

30 => Most of OSS CMS didn’t support the ability to decide the order & number of components - There are many useful CMS frameworks available - We tried to use these ﬁrst Why We Created From Scratch

Slide 30

Slide 30 text

31 How We Created the CMS

Slide 31

Slide 31 text

32 User Documentation When creating a CMS, we need to think about many things... How We Created the CMS Access Control Authorization Change History / Track Changes Security SSR Preview Status of Content

Slide 32

Slide 32 text

33 Wanted to release it as MVP - only support these features: - Provide enough components to create an existing page - Able to upload to staging environment via CMS to do QA - Able to upload to production environment via CMS to publish - Able to unpublish the page via CMS - Only accessible from in-house wiﬁ How We Created the CMS

Slide 33

Slide 33 text

34 Architecture

Slide 34

Slide 34 text

35 Architecture CMS Amazon S3 Cloud Functions Cloud Storage (Production) Cloud Storage (Staging) Host HTML, JS (React) ﬁle for CMS Campaign Page (Production) Campaign Page (Staging) Campaigns Data (JSON) & Static Files (HTML, JS) Campaigns Data (JSON) & Static Files (HTML, JS)

Slide 35

Slide 35 text

36 Architecture CMS Amazon S3 Cloud Functions Cloud Storage (Production) Cloud Storage (Staging) Provide HTML, JS ﬁle Get Data from Storage Host HTML, JS (React) ﬁle for CMS Campaign Page (Production) Campaign Page (Staging) Campaigns Data (JSON) & Static Files (HTML, JS) Campaigns Data (JSON) & Static Files (HTML, JS)

Slide 36

Slide 36 text

37 Architecture CMS Amazon S3 Cloud Functions Campaign Page (Production) Campaign Page (Staging) Host HTML, JS (React) ﬁle for CMS Cloud Storage (Production) Cloud Storage (Staging) Get Data from Storage Get Data from Storage Campaigns Data (JSON) & Static Files (HTML, JS) Campaigns Data (JSON) & Static Files (HTML, JS)

Slide 37

Slide 37 text

38 Architecture CMS Amazon S3 Cloud Functions Campaign Page (Production) Campaign Page (Staging) Host HTML, JS (React) ﬁle for CMS getData From Storage getData From Storage Cloud Storage (Production) Cloud Storage (Staging) 1. Get HTML file 2. Get base JS file 3. Get JSON file - including which component should be displayed 4. Get JS component files Campaigns Data (JSON) & Static Files (HTML, JS) Campaigns Data (JSON) & Static Files (HTML, JS)

Slide 38

Slide 38 text

39 Architecture CMS Amazon S3 Cloud Functions Campaign Page (Production) Campaign Page (Staging) Host HTML, JS (React) ﬁle for CMS Save as Draft Post Campaign Data & Images Cloud Storage (Production) Cloud Storage (Staging) Campaigns Data (JSON) & Static Files (HTML, JS) Campaigns Data (JSON) & Static Files (HTML, JS)

Slide 39

Slide 39 text

40 Architecture CMS Amazon S3 Cloud Functions Campaign Page (Production) Campaign Page (Staging) Host HTML, JS (React) ﬁle for CMS Publish Copy data to Production Cloud Storage (Production) Cloud Storage (Staging) Campaigns Data (JSON) & Static Files (HTML, JS) Campaigns Data (JSON) & Static Files (HTML, JS)

Slide 40

Slide 40 text

41 Campaigns Data (JSON) & Static Files (HTML, JS) Campaigns Data (JSON) & Static Files (HTML, JS) Architecture CMS Amazon S3 Cloud Functions Cloud Storage (Production) Cloud Storage (Staging) Campaign Page (Production) Campaign Page (Staging) Host HTML, JS (React) ﬁle for CMS Save as Draft Post campaign data and images Why are we using both and ?

Slide 41

Slide 41 text

42 Campaigns Data (JSON) & Static Files (HTML, JS) Campaigns Data (JSON) & Static Files (HTML, JS) Architecture CMS Amazon S3 Cloud Functions Cloud Storage (Production) Cloud Storage (Staging) Campaign Page (Production) Campaign Page (Staging) Host HTML, JS (React) ﬁle for CMS Save as Draft Post campaign data and images Do we need Cloud Function?

Slide 42

Slide 42 text

43 Architecture CMS Campaign Page (Production) Campaign Page (Staging) Authentication Service Authorization Database Google App Engine Cloud Storage (Production) Cloud Storage (Staging) Campaigns Data (JSON) & Static Files (HTML, JS) Campaigns Data (JSON) & Static Files (HTML, JS)

Slide 43

Slide 43 text

44 Architecture CMS Amazon S3 Cloud Functions Host HTML, JS (React) ﬁle for CMS Campaign Page (Production) Campaign Page (Staging) Cloud Storage (Production) Cloud Storage (Staging) Campaigns Data (JSON) & Static Files (HTML, JS) Campaigns Data (JSON) & Static Files (HTML, JS)

Slide 44

Slide 44 text

45 Codebase

Slide 45

Slide 45 text

46 Architecture CMS Amazon S3 Cloud Functions Campaign Page (Production) Campaign Page (Staging) Host HTML, JS (React) ﬁle for CMS Cloud Storage (Production) Cloud Storage (Staging) Campaigns Data (JSON) & Static Files (HTML, JS) Campaigns Data (JSON) & Static Files (HTML, JS)

Slide 46

Slide 46 text

47 Campaigns Data (JSON) & Static Files (HTML, JS) Campaigns Data (JSON) & Static Files (HTML, JS) Architecture CMS Amazon S3 Cloud Functions Campaign Page (Production) Campaign Page (Staging) Host HTML, JS (React) ﬁle for CMS CMS Cloud Storage (Production) Cloud Storage (Staging) creation-tool

Slide 47

Slide 47 text

48 Architecture

Slide 48

Slide 48 text

49 CMS

Slide 49

Slide 49 text

50 CMS

Slide 50

Slide 50 text

51 CMS

Slide 51

Slide 51 text

52 CMS

Slide 52

Slide 52 text

53 CMS

Slide 53

Slide 53 text

54 CMS

Slide 54

Slide 54 text

55 Launch

Slide 55

Slide 55 text

56 After Launching CMS - Authentication - Access Control - SSR

Slide 56

Slide 56 text

57 Re-Architecture CMS Campaign Page (Production) Campaign Page (Staging) Google App Engine Cloud Storage (Production) Cloud Storage (Staging) Auth0 Cloud DataStore

Slide 57

Slide 57 text

58 CMS Campaign Page (Production) Campaign Page (Staging) Google App Engine Cloud Storage (Production) Cloud Storage (Staging) Auth0 Cloud DataStore Re-Architecture

Slide 58

Slide 58 text

59 CMS Campaign Page (Production) Campaign Page (Staging) Google App Engine Cloud Storage (Production) Cloud Storage (Staging) Auth0 Cloud DataStore Create static HTML for SSR Re-Architecture

Slide 59

Slide 59 text

60 Future...

Slide 60

Slide 60 text

61 Future Segmentation Service Conversion Kicker Activity Logger Automation Service CMS Tool Banner Tool ML Notification

Slide 61

Slide 61 text

62 Future Segmentation Service Conversion Kicker Activity Logger Automation Service CMS Tool Banner Tool ML Notification

Slide 62

Slide 62 text

63 Future Segmentation Service Conversion Kicker Activity Logger Automation Service CMS Tool Banner Tool ML Notification

Slide 63

Slide 63 text

64 Future Segmentation Service Conversion Kicker Activity Logger Automation Service CMS Tool Banner Tool ML Notification

Slide 64

Slide 64 text

65 Conclusion

Slide 65

Slide 65 text

66 We built the CMS step by step - Used cloud function ﬁrst, and replaced it with GAE for a quick launch We built the CMS to create campaign pages easily - Can actually run more than 5 campaigns a month now - Decreased time needed to write code for LP, and can now focus on more important/fun things Conclusion

Slide 66

Slide 66 text

68 Conclusion https://twitter.com/sottar_  https://github.com/sottar  https://www.linkedin.com/in/sota-ohara  https://about.mercari.com/en/ 

Slide 67

Slide 67 text

69 - React https://reactjs.org/ - TypeScript https://www.typescriptlang.org/ - styled-components https://www.styled-components.com/ - nodejs https://nodejs.org/en/ - jest https://jestjs.io/ - Google Cloud Platform https://cloud.google.com/ References