Deep Convolutional Food Styling

What does a good photo of food look like?
Can a machine tell?
Can we fix up photos of food using deep learning?

Transcript

1. %FFQ$POWPMVUJPOBM
   'PPE4UZMJOH
   !MVOBSEPH
   %FFQ$POWPMVUJPOBM
   'PPE4UZMJOH
   %FFQ$POWPMVUJPOBM
   'PPE4UZMJOH
   

   View Slide

2. ࣗݾ঺հ
   w ϨγΣοΫ
   w HJUIVC!MVOBSEPH
   w ϙʔϥϯυਓ
   w ΫοΫύουͰݚڀ։ൃ෦
   w ػցֶशϦαʔνϟʔ
   w ೲ౾৯΂ΒΕΔ
   

   View Slide

3. recipes and
   photos by users
   

   View Slide

4. ྉཧࣸਅΛ
   ඒຯͦ͠͏ʹࡱΔ
   

   View Slide

5. ޫ
   https://snappygoat.com
   

   View Slide

6. https://snappygoat.com
   ৭
   

   View Slide

7. https://snappygoat.com
   ίϯτϥετ
   

   View Slide

8. https://snappygoat.com
   ߏਤ
   

   View Slide

9. https://snappygoat.com
   ޫ୔
   

   View Slide

10. https://snappygoat.com
    എܠ
    

    View Slide

11. https://snappygoat.com
    ಁ໌ੑ
    

    View Slide

12. https://snappygoat.com
    ཱମੑ
    

    View Slide

13. ɻɻɻ
    

    View Slide

14. 5VSJOH5FTU
    

    View Slide

15. νϡʔϦϯάɾςετ
    

    View Slide

16. View Slide

17. 5VSJOH5FTU
    G
    D
    

    View Slide

18. 5VSJOH5FTU
    哋▶
    
    

    View Slide

19. 5VSJOH5FTU
    Generative Adversarial Learning
    Generator
    Discriminator
    (
    %
    

    View Slide

20. View Slide

21. %FFQ
    $POWPMVUJPOBM
    (FOFSBUJWF
    "EWFSTBSJBM
    /FUXPSLT
    

    View Slide

22. %PXOTBNQMJOH
    features or
    small resolution
    image
    convolutional
    layer or layers
    RGB image
    input
    output
    

    View Slide

23. 6QTBNQMJOH
    upsampling
    CNN layer
    or layers
    RGB image
    features or
    small resolution
    image
    input
    output
    

    View Slide

24. &ODPEFS%FDPEFS
    D
    E
    image in
    Decoder
    Encoder
    image out
    feature space
    

    View Slide

25. 'VMMZ$POOFDUFE
    $MBTTJpFS
    YES
    NO
    classes
    data or
    features
    also called “Dense” 

    or “Linear” layer
    

    View Slide

26. (
    %
    νϡʔϦϯάɾςετษڧձ
    

    View Slide

27. Generator
    Discriminator
    G
    MPPLTMFHJU
    UPUBMMZTIPQQFE
    D
    

    View Slide

28. G SFBM
    GBLF
    D
    D(G(noise)) ˠ real
    (FOFSBUPSUSBJOJOH
    Discriminator acts
    as the teacher
    

    View Slide

29. G SFBM
    GBLF
    D
    SFBM
    GBLF
    D
    D(G(noise)) ˠ fake
    D(photo) ˠ real
    %JTDSJNJOBUPSUSBJOJOH
    Generator provides
    negative examples
    

    View Slide

30. View Slide

31. $ZDMF("/
    

    View Slide

32. w QZUPSDIIUUQTKVOZBO[HJUIVCJP$ZDMF("/
    w UFOTPSqPXIUUQTHJUIVCDPNWBOIVZ[$ZDMF("/5FOTPS'MPX
    

    View Slide

33. https://www.youtube.com/watch?v=9reHvktowLY
    

    View Slide

34. https://github.com/tatsuyah/CycleGAN-Models
    

    View Slide

35. w DIBJOFSIUUQTHJUIVCDPN"JYJMFDIBJOFSDZDMFHBO
    

    View Slide

36. (FOFSBUPS
    transformation network
    https://arxiv.org/abs/1603.08155
    

    View Slide

37. TVCUMF
    WJWJE
    …
    TVCUMF
    WJWJE
    %JTDSJNJOBUPS
    fully convolutional
    judges patches 

    of the input image
    https://arxiv.org/abs/1603.08155
    

    View Slide

38. "EWFSTBSJBM-PTT
    X
    F
    G Y
    TVCUMF
    WJWJE
    TVCUMF
    WJWJE
    …
    TVCUMF
    WJWJE
    TVCUMF
    WJWJE
    …
    X(F(low_score)) ˠ classify as high_score
    Y(F(high_score)) ˠ classify as low_score
    

    View Slide

39. $ZDMF-PTT
    G F
    G(F(image))ˠ the same image
    F G
    F(G(image))ˠ the same image
    

    View Slide

40. 5SBJOJOH
    

    View Slide

41. subtle vivid
    

    View Slide

42. View Slide

43. View Slide

44. View Slide

45. View Slide

46. View Slide

47. View Slide

48. View Slide

49. View Slide

50. View Slide

51. View Slide

52. View Slide

53. View Slide

54. *OUIFXJME
    

    View Slide

55. View Slide

56. View Slide

57. View Slide

58. View Slide

59. View Slide

60. View Slide

61. View Slide

62. View Slide

63. View Slide

64. View Slide

65. View Slide

66. View Slide

67. View Slide

68. 'BJM
    

    View Slide

69. View Slide

70. View Slide

71. View Slide

72. View Slide

73. View Slide

74. View Slide

75. View Slide

76. View Slide

77. View Slide

78. 6O*OTUB'JMUFS
    

    View Slide

79. https://snappygoat.com
    

    View Slide

80. https://snappygoat.com
    

    View Slide

81. https://snappygoat.com
    

    View Slide

82. Ҏ্
    

    View Slide