Super Resolution with CoreML (Long Version)

Super Resolution with CoreML (Long Version)

This is the slide presented at try! Swift NewYork 2018.
https://www.tryswift.co/events/2018/nyc/#coreml

The ‘Super Resolution' technique is used for converting low resolution images into high resolution, which reduces the amount of image data that needs to be transferred. In this talk, I'd like to show you the implementation of super resolution with CoreML and Swift, and compare the results with conventional methods. I’ll also talk about how to train your own model using your own data step by step. In addition, I’d like to introduce recent topics such as Turi Create, Swift for TensorFlow, CoreML2 and CreateML, which will be enhanced in iOS 12 and I’ll look at how iOS will take advantage of machine learning technology in the future.

D2f212ce418f3daa29c23914c9b6892b?s=128

Kentaro Matsumae

September 05, 2018
Tweet

Transcript

  1. 12.
  2. 14.
  3. 29.

    $POW-BZFS $POW-BZFS $POW-BZFS 4VQFS3FTPMVUJPOXJUI$// 43$// ʜ ʜ ʜ ʜ ϒϥοΫδϟοΫʹΑΖ͘͠

    ࠤ౻लๆ Give My Regards to Black Jack SHUHO SATO *OQVU -PX3FTPMVUJPO
  4. 30.

    0VUQVU )JHI3FTPMVUJPO $POW-BZFS $POW-BZFS $POW-BZFS 4VQFS3FTPMVUJPOXJUI$// 43$// ʜ ʜ ʜ

    ʜ ϒϥοΫδϟοΫʹΑΖ͘͠ ࠤ౻लๆ Give My Regards to Black Jack SHUHO SATO *OQVU -PX3FTPMVUJPO
  5. 31.

    0VUQVU )JHI3FTPMVUJPO *NBHF $POW-BZFS $POW-BZFS $POW-BZFS 4VQFS3FTPMVUJPOXJUI$// 43$// ʜ ʜ

    ʜ ʜ ϒϥοΫδϟοΫʹΑΖ͘͠ ࠤ౻लๆ Give My Regards to Black Jack SHUHO SATO
  6. 32.

    0VUQVU )JHI3FTPMVUJPO *NBHF $POW-BZFS $POW-BZFS $POW-BZFS 4VQFS3FTPMVUJPOXJUI$// 43$// ʜ ʜ

    ʜ ʜ ϒϥοΫδϟοΫʹΑΖ͘͠ ࠤ౻लๆ Give My Regards to Black Jack SHUHO SATO
  7. 33.

    0VUQVU )JHI3FTPMVUJPO *NBHF $POW-BZFS $POW-BZFS $POW-BZFS 4VQFS3FTPMVUJPOXJUI$// 43$// ʜ ʜ

    ʜ ʜ ϒϥοΫδϟοΫʹΑΖ͘͠ ࠤ౻लๆ Give My Regards to Black Jack SHUHO SATO
  8. 34.

    0VUQVU )JHI3FTPMVUJPO *NBHF $POW-BZFS $POW-BZFS $POW-BZFS 4VQFS3FTPMVUJPOXJUI$// 43$// ʜ ʜ

    ʜ ʜ ϒϥοΫδϟοΫʹΑΖ͘͠ ࠤ౻लๆ Give My Regards to Black Jack SHUHO SATO
  9. 35.
  10. 56.

    43$//.PEFM4USVDUVSF <  > <  > <  

    > <   > <   > ϒϥοΫδϟοΫʹΑΖ͘͠ ࠤ౻लๆ Give My Regards to Black Jack SHUHO SATO
  11. 57.

    43$//.PEFM4USVDUVSF <  > <  > <  

    > <   > <   >  QBSBNFUFST ϒϥοΫδϟοΫʹΑΖ͘͠ ࠤ౻लๆ Give My Regards to Black Jack SHUHO SATO
  12. 58.

    5SBJOJOHDPEF ch = 1 SRCNN = Sequential(input_shape=(200, 200, ch)) SRCNN.add(Conv2D(128,

    9, 9, activation='relu')) SRCNN.add(Conv2D(64, 3, 3, activation='relu')) SRCNN.add(Conv2D(ch, 5, 5, activation='linear')) adam = Adam(lr=0.0003) SRCNN.compile(optimizer=adam, loss='mean_squared_error') SRCNN.fit(..) SRCNN.save(model_dir, 'model.h5')
  13. 66.
  14. 67.

    '43$//QBSBNFUFST ʜ *OQVU -3*NBHF 0VUQVU )3*NBHF ʜ ʜ ʜ ʜ

    ʜ ʜ <   > <   > <   > <   > <   > <   > <   > <   > ϒϥοΫδϟοΫʹΑΖ͘͠ ࠤ౻लๆ Give My Regards to Black Jack SHUHO SATO
  15. 68.

    43$// QBSBNFUFST '43$//QBSBNFUFST ʜ *OQVU -3*NBHF 0VUQVU )3*NBHF ʜ ʜ

    ʜ ʜ ʜ ʜ <   > <   > <   > <   > <   > <   > <   > <   > ϒϥοΫδϟοΫʹΑΖ͘͠ ࠤ౻लๆ Give My Regards to Black Jack SHUHO SATO
  16. 69.

    43$// QBSBNFUFST '43$//QBSBNFUFST ʜ *OQVU -3*NBHF 0VUQVU )3*NBHF ʜ ʜ

    ʜ ʜ ʜ ʜ <   > <   > <   > <   > <   > <   > <   > <   > '43$// QBSBNFUFST ϒϥοΫδϟοΫʹΑΖ͘͠ ࠤ౻लๆ Give My Regards to Black Jack SHUHO SATO
  17. 70.

    Performance #JDVCJD 43$// '43$// 1SFEJDUJPO 5JNFQBHF TFD TFD TFD 5SBJOJOH

    5JNF  IPVST IPVST .-.PEFM 4J[F  ,# ,# 14/3 EC EC EC
  18. 76.

    Easy to use let imageView: UIImageView = … let image:

    UIImage = … imageView.setSRImage(image) //Super Resolution⭐ pod “SuperResolutionKit”
  19. 92.

    $IPPTFUIFCFTUXBZ 5FDIOPMPHZ (PBMT -FWFM 7JTJPOGSBNFXPSL 'BDF5FYU%FUFDUJPO  &BTZ $SFBUF.- $VTUPN*NBHF5FYU$MBTTJpDBUJPO

     &BTZ 5VSJ$SFBUF $VTUPN0CKFDU%FUFDUJPO 4UZMF5SBOTGFS  .JE $PSF.- :PVSPXONPEFM 0UIFST $VUUJOHFEHFNFUIPET )BSE
  20. 98.

    from keras.models import Sequential from keras.layers import Conv2D from keras.optimizers

    import Adam ch = 1 SRCNN = Sequential(input_shape=(200, 200, ch)) SRCNN.add(Conv2D(128, 9, 9, activation='relu')) SRCNN.add(Conv2D(64, 3, 3, activation='relu')) SRCNN.add(Conv2D(ch, 5, 5, activation='linear')) adam = Adam(lr=0.0003) SRCNN.compile(optimizer=adam, loss='mean_squared_error')
  21. 99.

    let Sequential = Python.import("keras.models.Sequential") let Conv2D = Python.import("keras.layers.Conv2D") let Adam

    = Python.import("keras.optimizers.Adam") let ch = 1 let SRCNN = Sequential(input_shape: (200, 200, ch)) SRCNN.add(Conv2D(128, (9, 9), activation: 'relu')) SRCNN.add(Conv2D(64, (3, 3), activation: 'relu')) SRCNN.add(Conv2D(ch, (5, 5), activation: 'linear')) let adam = Adam(lr: 0.0003) SRCNN.compile(optimizer: adam, loss: 'mean_squared_error') %ZOBNJD.FNCFS-PPLVQ