Sharing experience of LINE Infrastructure

Slide 1

Slide 1 text

No content

Slide 2

Slide 2 text

Park Younghee LINE Executive Officer Head of ITService Center

Slide 3

Slide 3 text

COVID-19 is redefining IT infrastructure

Slide 4

Slide 4 text

COVID-19 is redefining IT infrastructure Our preparation was “not enough”

Slide 5

Slide 5 text

LINE User 0 50000 100000 150000 200000 250000 300000 350000 400000 450000 2011/6/1 2011/08/23/ 2011/10/14/ 2011/10/17/ 2011/11/08/ 2011/11/29/ 2012/01/27/ 2012/03/05/ 2012/03/27/ 2012/04/18/ 2012/06/06/ 2012/10/25/ 2012/11/30/ 2013/05/01/ 2013/11/25/ 2014/04/02/ ※The number of downloads until June 2011, and the number of registered users after March 2012. Winter is coming

Slide 6

Slide 6 text

LINE MAU ※MAU (4 major countries) 0 10000 20000 30000 40000 50000 60000 70000 80000 90000 2014.2Q 2014.4Q 2015.2Q 2015.4Q 2016.2Q 2016.4Q 2017.2Q 2017.4Q 2018.2Q 2018.4Q 2019.2Q 2019.4Q 2020.2Q Network x 6 CDN x 12 Server x 5 Storage x 4 IDC JP*2, SG, US, DE, KR, TW*2 Office x3 Infrastructure

Slide 7

Slide 7 text

Speed Global Scale

Slide 8

Slide 8 text

• Problem: difficulties of understanding current status of infrastructure Period of rapid growth of messaging

Slide 9

Slide 9 text

• Problem: difficulties of understanding current status of infrastructure • LINE engineers need to understand event alarm, server fault latency of database, long query traffic of L4/L7 load balance DNS setting, certificate setting blocking by network, firewall ACL inter IDC traffic, ISP traffic Period of rapid growth of messaging

Slide 10

Slide 10 text

Infrastructure Information Dashboard - Infra Management Console (IMC)

Slide 11

Slide 11 text

Infrastructure Information Dashboard - Infra Management Console (IMC) Network Traffic CMDB Server Monitoring SSL Cert Load Balancer IMC DB Monitoring ACL

Slide 12

Slide 12 text

Transparency of information is the start of collaboration.

Slide 13

Slide 13 text

• Problem: late delivery, operation in large scale Period of emergence of new services as Life Portal

Slide 14

Slide 14 text

• Problem: late delivery, operation in large scale We decided changing workflow to cloud • OpenStack maturity • Change in motivation Work as a developer to solve user’s problem Implement new features to user Provide values to end users and App developers • Cost consideration Period of emergence of new services as Life Portal

Slide 15

Slide 15 text

vm baremetal SSO OpenStack IaaS

Slide 16

Slide 16 text

vm baremetal SSO OpenStack IaaS

Slide 17

Slide 17 text

Verda is a Private Cloud for LINE is based on OpenStack. since 2016~ FaaS PaaS IaaS NAT Load Balancer Baremetal

Slide 18

Slide 18 text

Verda is a Private Cloud for LINE is based on OpenStack. since 2016~ FaaS PaaS IaaS NAT Load Balancer Baremetal Hypervisor 2,000+ Virtual Machine 55,000+ Physical Machine 20,000+

Slide 19

Slide 19 text

Verda is a Private Cloud for LINE

Slide 20

Slide 20 text

Verda is a Private Cloud for LINE

Slide 21

Slide 21 text

Easy way to save infrastructure cost

Slide 22

Slide 22 text

Easy way to save infrastructure cost Define low usage server Set about 15% line by cpu, mem, filesystem, network 1 High utilization 85% low utilization 15% Low usage server All barematal & vm

Slide 23

Slide 23 text

Easy way to save infrastructure cost Define low usage server Set about 15% line by cpu, mem, filesystem, network 1 Decrease low usage servers • return • baremetal à vm • Spec down • containerizing • change architecture 2 High utilization 85% low utilization 15% Low usage server All barematal & vm

Slide 24

Slide 24 text

Easy way to save infrastructure cost Define low usage server Set about 15% line by cpu, mem, filesystem, network 1 Decrease low usage servers • return • baremetal à vm • Spec down • containerizing • change architecture 2 Strict standard when low usage server < 10% 3 High utilization 85% low utilization 15% Low usage server All barematal & vm

Slide 25

Slide 25 text

Next challenge : Productivity in both of Dev, Infrastructure

Slide 26

Slide 26 text

Next challenge : Productivity in both of Dev, Infrastructure • Easily movable application (operation) • Declarative use of infrastructure & platform (ease of use) • Utilize resources of entire datacenter (cost optimization)

Slide 27

Slide 27 text

Technology needs to understand the business

Slide 28

Slide 28 text

Project Plano Physical/Virtual Machines Applications Development Application Operation Deployment Infrastructure Operations App Developers Infrastructure Operators AS-IS Responsibilities

Slide 29

Slide 29 text

Project Plano Resource Abstraction Physical/Virtual Machines Applications TO-BE Responsibilities Physical/Virtual Machines Applications Development Application Operation Deployment Infrastructure Operations App Developers Infrastructure Operators AS-IS Responsibilities

Slide 30

Slide 30 text

Responsibility as a social infrastructure • BCP/DR • IPv6

Slide 31

Slide 31 text

BCP/DR “text messaging”

Slide 32

Slide 32 text

1st IDC 2nd IDC 7ms, 500km Over 100 Gbps Message front Message backend Redis Hbase Message front internet SSL VPN DNS NTP Kerberos JIRA Wiki Git

Slide 33

Slide 33 text

1st IDC 2nd IDC 7ms, 500km Over 100 Gbps Message front Message backend Redis Hbase Message front internet SSL VPN DNS NTP Kerberos JIRA Wiki Git SSL VPN DNS NTP Kerberos JIRA Wiki Git

Slide 34

Slide 34 text