Upgrade to Pro — share decks privately, control downloads, hide ads and more …

PicCell : Flooding your service with AI

PicCell : Flooding your service with AI

Joonsik Baek
LINE Plus LPE4 MediaPlatform Software Engineer
https://linedevday.linecorp.com/jp/2019/sessions/F1-6

Be4518b119b8eb017625e0ead20f8fe7?s=128

LINE DevDay 2019

November 20, 2019
Tweet

Transcript

  1. 2019 DevDay PicCell : Flooding Your Service With AI >

    Joonsik Baek > LINE Plus LPE4 MediaPlatform Software Engineer
  2. Flood Your Service With AI 2

  3. Flood Your Service With AI 3

  4. 4

  5. 5 ?

  6. 6 !

  7. 7 > Why and How PicCell Was Made > Case

    Studies > Roadmap
  8. 8 Why PicCell Was Made

  9. 9 Why PicCell Was Made

  10. 10 Low ROI Why PicCell Was Made

  11. 11 ??? Why PicCell Was Made

  12. 12 ??? Why PicCell Was Made

  13. 13 Why PicCell Was Made

  14. 14 Why PicCell Was Made

  15. 15 More Than
 40 Services Why PicCell Was Made

  16. 16 Developed by ???

  17. 17 Media Platform Who Is Qualified

  18. 18 Media Platform Who Is Qualified

  19. 19 Media Platform Who Is Qualified - More Than 80

    Services
  20. 20 Media Platform OBSObject Storage Who Is Qualified - More

    Than 80 Services
  21. 21 Media Platform Video Image Audio File Who Is Qualified

    OBSObject Storage - More Than 80 Services
  22. 22 Media Platform Video Image Audio File Storage Delivery Processing

    Who Is Qualified OBSObject Storage - More Than 80 Services
  23. 23 Media Platform Video Image Audio File Storage Delivery Processing

    Who Is Qualified OBSObject Storage - More Than 80 Services - MAX 1.5 Tbps Traffic - 100 PB Storage
  24. 24 Media Platform Video Image Audio File Storage Delivery Processing

    Who Is Qualified OBSObject Storage - More Than 80 Services - MAX 1.5 Tbps Traffic - 100 PB Storage
  25. 25 Media Platform Video Image Audio File Storage Delivery Processing

    Who Is Qualified OBSObject Storage - More Than 80 Services - MAX 1.5 Tbps Traffic - 100 PB Storage
  26. 26 ? ? ?

  27. 27 ? ? Adult
 Content
 Filter ?

  28. ResNet 50
 Based 28 Adult Content Filter

  29. 3 Classes
 (Adult, Sexy, Normal) ResNet 50
 Based 29 Adult

    Content Filter
  30. 3.5M
 Training Samples ResNet 50
 Based 30 Adult Content Filter

    3 Classes
 (Adult, Sexy, Normal)
  31. > 0.9 F1 Score 3.5M
 Training Samples ResNet 50
 Based

    31 Adult Content Filter 3 Classes
 (Adult, Sexy, Normal) 1.1M Test Set
  32. 300ms
 Processing Time > 0.9 F1 Score 3.5M
 Training Samples

    ResNet 50
 Based 32 Adult Content Filter 3 Classes
 (Adult, Sexy, Normal) 1.1M Test Set
  33. 33 18 Adult Content Filter

  34. 34 18 " " " "" " Adult Content Filter

  35. 35 18

  36. 36 18

  37. 37 18 Sync Async Gateway

  38. 38 18 Sync Async Gateway Pacman

  39. 39 18 Sync Async Gateway Pacman Realtime Processing

  40. 40 Req. #1 Req. #2 Req. #3 Callback. #1 Callback.

    #3 Guaranteed Callback Callback. #2
  41. 41 Req. #1 Req. #2 Req. #3 Callback. #1 Callback.

    #2 Callback. #3 Guaranteed Callback
  42. 42 Req. #1 Req. #2 Req. #3 Callback. #1 Callback.

    #2 Callback. #3 Guaranteed Callback #
  43. 43 Consumer Publisher Message Queue Guaranteed Callback

  44. 44 Consumer Publisher Message Queue Ack Guaranteed Callback

  45. 45 Consumer Publisher Message Queue Nack Guaranteed Callback

  46. 46 Consumer Publisher Message Queue Dead-Letter Queue
 With TTL Nack

    Guaranteed Callback
  47. 47 18 Sync Async Gateway Pacman DLQ Spiderman $ Guaranteed


    Callback
  48. 48 Result
 Message Queue Dead-Letter Queue
 With TTL Nack Guaranteed

    Callback Pacman Spiderman $ Guaranteed
 Callback
  49. 49 Expensive Duplicated Processing

  50. 50 Expensive Duplicated Processing

  51. 51 GPU Expensive Duplicated Processing

  52. 52 % = GPU Expensive Duplicated Processing

  53. 53 18 Sync Async Gateway Pacman DLQ Spiderman $ Guaranteed


    Callback
  54. 54 Gateway DLQ Search Result Write Request Process Write Result

    Req. For Deliver Result
  55. 55 Gateway DLQ Search Result Write Request … Search Result

    Process Write Result Req. For Req. For … Deliver Result Deliver Result
  56. 56 Image Adult Content Filter for Video

  57. 57 Image Video Adult Content Filter for Video

  58. 58 ??? Image Video Adult Content Filter for Video

  59. 59 Adult Content Filter for Video

  60. 60 JPG JPG JPG Adult Content Filter for Video

  61. 61 JPG JPG JPG {;} {;} {;} {;} Adult Content

    Filter for Video
  62. 62 Timberman

  63. 63 18 Sync Async Gateway Pacman DLQ Spiderman $ Timberman

    Guaranteed
 Callback
  64. 64 Image Video Adult Content Filter for Video

  65. 65 The Most Pleasant LINE Ever

  66. 66 Illegal Advertisements in Timeline

  67. 67 Illegal Advertisement Adult Content Illegal Advertisements in Timeline

  68. 68 Illegal Advertisement Adult Content Illegal Advertisements in Timeline

  69. 69 ? Adult
 Content
 Filter Illegal
 Ad. Detection ?

  70. 70 Illegal
 Ad. Illegal Advertisements in Timeline Imitation Product QR

    Code Advertising Text on Image
  71. 71 Illegal Advertisements in Timeline Imitation Product QR Code Advertising

    Text on Image Illegal
 Ad.
  72. 72 Illegal Advertisements in Timeline Imitation Product QR Code Advertising

    Text on Image Illegal
 Ad.
  73. 73 Illegal Advertisements in Timeline Imitation Product QR Code Advertising

    Text on Image Illegal
 Ad.
  74. 74 Illegal Advertisements in Timeline Imitation Product QR Code Advertising

    Text on Image ✅ Product Classification - Sunglasses, Shoes, Watch or Handbag Illegal
 Ad.
  75. 75 Illegal Advertisements in Timeline Imitation Product QR Code Advertising

    Text on Image ✅ Product Classification - Sunglasses, Shoes, Watch or Handbag Illegal
 Ad.
  76. 76 Illegal Advertisements in Timeline Imitation Product QR Code Advertising

    Text on Image ✅ Product Classification - Sunglasses, Shoes, Watch or Handbag ✅ QR Code Reader - URL Extraction Illegal
 Ad.
  77. 77 Illegal Advertisements in Timeline Imitation Product QR Code Advertising

    Text on Image ✅ Product Classification - Sunglasses, Shoes, Watch or Handbag ✅ QR Code Reader - URL Extraction ✅ OCR (Optical Character Recognition) Illegal
 Ad.
  78. 78 Illegal Advertisements in Timeline Imitation Product QR Code Advertising

    Text on Image ✅ Product Classification - Sunglasses, Shoes, Watch or Handbag ✅ QR Code Reader - URL Extraction ✅ OCR (Optical Character Recognition) - JP / KR / EN / TH / TW / CN / ID Illegal
 Ad.
  79. 79 18 Sync Async Gateway Pacman DLQ Spiderman $ Timberman

    ' ( ⌚ * Guaranteed
 Callback
  80. 80 Illegal Ad Free

  81. 81 Trial and Error via LINE LIVE

  82. 18 82 Trial and Error via LINE LIVE

  83. 18 83 Trial and Error via LINE LIVE - Gun

    - Knife - Cigarette - Electric Cigarette - Sensitive Symbols
  84. 84 Trial and Error via LINE LIVE - Gun -

    Knife - Cigarette - Electric Cigarette - Sensitive Symbols + 18
  85. 85 18 ?? Trial and Error via LINE LIVE +

    - Gun - Knife - Cigarette - Electric Cigarette - Sensitive Symbols
  86. 86 18 ?? Trial and Error via LINE LIVE +

    - Gun - Knife - Cigarette - Electric Cigarette - Sensitive Symbols F1 Score 0.9 0.3
  87. 87 18 Service Specific Training

  88. 88 18 Service Specific Training

  89. 89 JPG JPG JPG JPG JSON JSON JSON JSON Service

    Specific Training
  90. 90 JPG JPG JPG JPG JSON JSON JSON JSON -

    User Consent - Disciplined Service Specific Training
  91. 91 JPG JPG JPG JPG JSON JSON JSON JSON Content

    Deep-Learning Engineer Service Specific Training Training Sample
 Gateway Legal Review +
  92. 92 JPG JPG JPG JPG JSON JSON JSON JSON Content

    DL Engineer Access-Token Service Specific Training Training Sample
 Gateway
  93. 93 Librarian

  94. 94 18 Sync Async ' ( ⌚ * Gateway Spiderman

    $ Timberman DLQ + Pacman Librarian Access
 Token
  95. F1 Score 0.3 0.84 40,000
 Training Samples 3,000 Training Samples

    95 Service Specific Training
  96. Back-Office Support Adult Content Filter Object Detection … OCR -

    JP/EN/CN/TW/ID/KR AD Detection 96 …
  97. Back-Office Support Adult Content Filter Object Detection 15M / Day

    … OCR - JP/EN/CN/TW/ID/KR AD Detection 97 …
  98. Back-Office Support Adult Content Filter Object Detection 15M / Day

    1.3M / Day 15M / Day 3.5M / Day … OCR - JP/EN/CN/TW/ID/KR AD Detection 98 …
  99. 99 0 80000 160000 240000 320000 400000 2017 2018 2019

    Daily Monitored Images by Operators Monitoring Cost Reduction
  100. 100 0 80000 160000 240000 320000 400000 2017 2018 2019

    Daily Monitored Images by Operators Monitoring Cost Reduction
  101. Case Studies 101

  102. 102 Closing the Distance

  103. 103 CLOSING THE DISTANCE LINEΛ௨ͯ͡ɺੈքதͷਓͱਓɺ ਓͱ৘ใɾαʔϏεͱͷڑ཭ΛॖΊ·͢ LINE Closing the Distance

  104. 104 Translation Infra. X Closing the Distance

  105. 105 Closing the Distance ,

  106. 106 Closing the Distance , - . / 0 1

    2 1 —
 6
  107. 107 Closing the Distance , 2 1 0 . -

    1 -
  108. 108 Closing the Distance , 2 1 0 . -

    1 1 —
 2 1 —
 3 1 —
 2 -
  109. 109 Closing the Distance

  110. 110 Closing the Distance

  111. 111 Closing the Distance

  112. 112 Closing the Distance

  113. 113 Closing the Distance

  114. 114 18 Sync Async ' ( ⌚ * Gateway Librarian

    Spiderman $ Timberman DLQ + Pacman Access
 Token
  115. 115 Expansion Into Finance LINE FINANCE

  116. 116 Expansion Into Finance KYC 3 LINE FINANCE

  117. 117 eKYC KYC 3 ! Expansion Into Finance LINE FINANCE

  118. 118 eKYC KYC 3 ! Expansion Into Finance 444
 444

    LINE FINANCE
  119. Financial
 Solution 119 ? Adult
 Content
 Filter Illegal
 Ad. Detection

  120. 120 Expansion Into Finance LINE FINANCE eKYC

  121. ID card Recognition 121 - Jane Doe Expansion Into Finance

    LINE FINANCE eKYC
  122. ID card Recognition 122 - Jane Doe Expansion Into Finance

    LINE FINANCE eKYC
  123. ID card Recognition 123 - Jane Doe Expansion Into Finance

    LINE FINANCE eKYC
  124. ID card Recognition Face Verification 124 - Jane Doe 5

    + Expansion Into Finance LINE FINANCE eKYC
  125. ID card Recognition Face Verification 125 - Jane Doe 5

    + Expansion Into Finance x 0.5 Speed LINE FINANCE eKYC
  126. ID card Recognition Face Verification Secured Infra. (JP) 126 -

    Jane Doe 5 + Expansion Into Finance LINE FINANCE eKYC
  127. ID card Recognition Face Verification Secured Infra. (JP) 127 -

    Jane Doe 5 + Expansion Into Finance LINE FINANCE eKYC
  128. Roadmap 128

  129. Deep-Learning Training Sample Portal 129 Training Sample Portal

  130. Deep-Learning Training Sample Portal 130 JPG JPG JPG JPG JSON

    JSON JSON JSON Normal
 Librarian Deep-Learning Engineers
  131. Deep-Learning Training Sample Portal 131 JPG JPG JPG JPG JSON

    JSON JSON JSON Normal
 Librarian Deep-Learning Engineers Refine Training Sample With Correction API 
 Adult Correction API
  132. 132 Only Corrected Samples Only Samples From JP Training Sample

    Portal Deep-Learning Training Sample Portal Librarian
  133. 133 Only Corrected Samples Only Samples From JP Samples
 From


    JP Corrected
 Samples Training Sample Portal Deep-Learning Training Sample Portal Librarian
  134. Cloud Vision API Cognitive Service 134

  135. Cloud Vision API Cognitive Service 135

  136. Cloud Vision API Cognitive Service 136

  137. Cloud Vision API Cloud Vision API Cognitive Service 137

  138. 138

  139. 139

  140. Thank You for Your Attention 140