Save 37% off PRO during our Black Friday Sale! »

Learning to Generate Synthetic Data via Compositing

Db0553d2aacb394f95a0dd064d0311bf?s=47 phalanx
July 22, 2019
660

Learning to Generate Synthetic Data via Compositing

Db0553d2aacb394f95a0dd064d0311bf?s=128

phalanx

July 22, 2019
Tweet

Transcript

  1. Learning to Generate Synthetic Data via Compositing 画像処理&機械学習 論文LT会 #5

    @phalanx 22 July, 2019
  2. Self Introduction • Name: phalanx • Data Scientist at DeNA

    • Machine Learing: 1.5 year • Kaggle: 1 year • Kaggle Master • TGS 1st place • iMet 7th place • Petfinder 17th place • HPA 36th place @ZFPhalanx
  3. Outline • Task aware approach to synthetic data generation •

    Our pipeline consist of three components • Synthesizer Network: generate composite image • Target Network: classify/detect foreground object in composite image • Discriminator: identify whether composite image is real or not
  4. Outline • Limitation of prior approaches • Generating synthetic data

    is decoupled from training the target classifier → synthetic data has little value in improving performance of target network • Our approach • Synthesizer and target networks are trained in an adversarial manner → synthesizer produces meaningful training samples
  5. Synthesizer Network • Inputs: background image(), foreground object mask() •

    Output: transformation function() • Restrict A to set of 2D affine transformation in this paper • Composite synthetic image: = ۩ () • ۩: alpha blending • Sptial transformer network create by , ,
  6. Synthesizer Network: architecture • Shared Feature Network • Identical feature

    extraction on and • Foreground/Background branch • Identical mid-level feature extraction on and • FC Regression Network: • Concatnate mid-level feature of and • Outputs affine transformation parameter
  7. Target Network • Neural network trained for specific task(classification, detection,

    etc.) • Target network is fine-tuned with composite image • Loss function: • Image classification: cross entropy loss • Object detection: • classification: cross entropy, localization: smooth 1
  8. Discriminator • motivation • Realistic data can help the target

    network to learn more efficiently • Synthesizer need to produce realistic composite image • Binary classification • Input(composite image, real images) • Loss function : Ε log + Ε log 1 −
  9. Training • Train models according to , , • Update

    parameter of while keeping parameters of , fixed • Update parameter of , while keeping parameter of fixed : Synthesizer network, : Target network, : Discriminator
  10. Performance with AffNIST • AffNIST: transformed MNIST by randomly sampled

    affine transform • Red line: train model with MNIST, then finetune with AffNIST data • Green line: train model with MNIST, then finetune with Synthetic data • Synthesis data • foreground:MNIST digits • Background: black background
  11. Performance with Pascal VOC • Comparison of our approach with

    prior approach • Cut-Paste-Learn [7] • Context-Data-Augmentation [6] • Synthesis data • foreground:instance mask from voc2007/2012 • Background: coco Results on VOC 2007
  12. Performance with Pascal VOC • Quality of Synthetic Data •

    Our approach generate harder examples than prior approach
  13. Synthetic Network Output • Top: composite image without discriminator •

    Bottom: composite image with discriminator
  14. Thank you! Questions?