Design in the Age of Synthetic Realities

Presentation at UX Australia 2019

AI-generated and mixed realities are blurring the boundaries of “truth” and challenging how we value it. Synthetic Realities have reached new heights of sophistication, sparking controversy, and also fascination, about its creative possibilities.

Customers will increasingly expect brands to meet them halfway to supplement the realities they desire. Next, they will expect reality to adapt to them in real time without any conscious request. And the role of designers will be to set the stage on which these experiences happen, often curating AI systems rather than designing themselves.

UX practitioners face the double task of designing the interfaces to these new products, tools and services as well as seeing design for digital disrupted as radically as the invention of Photoshop and desktop publishing did three decades ago. If anyone can summon up an entirely generated image or an interface just by speaking its description, what is the future role of the designer?

The mixture of fear and fascination of the power of synthetic media is understandable but wrongly frames its future trajectory. This talk takes a view on synthetic realities not as something brand new, but the latest iteration of a long history of re-defining truth and reality through media manipulation.

Using visually compelling examples from the latest research and work in synthetic and generated media and charting their convergence, we'll see just how quickly they will be absorbed by culture, radically shake-up the fields of design, advertising and marketing in a very short time-frame, and transform the role of future designers and creatives. It raises important questions of design, ethics, and craft that the UX and broader design community will have to find answers to pretty damn quickly.

Transcript

1. Design in the age of
   synthetic realities
   Design and Innovation from
   Accenture Interactive
   Andy Polaine
   Group Design Director – Client Evolution
   @apolaine | @fjord
   Slides: pln.me/uxaus-synthetic
   UX Australia, Sydney, 2019
   

   View Slide

2. 2
   Andy Polaine – Design in the age of synthetic realities
   Synthetic realities
   trends.fjordnet.com
   

   View Slide

3. 3
   Andy Polaine – Design in the age of synthetic realities
   Image source: Adrian on Unsplash
   Jordan Peele’s Obama PSA
   

   View Slide

4. 4
   Andy Polaine – Design in the age of synthetic realities
   Deep Fakes
   

   View Slide

5. 5
   Andy Polaine – Design in the age of synthetic realities
   Bundesweite Umfrage
   Jetzt Umfrage beantworten und exklusives Dankeschön-Paket sichern
   MDM ÖFFNEN
   The Post's View
   Deepfakes are dangerous — and they target a huge
   weakness
   Opinion
   •
   June 16
   DEEPFAKES ARE dangerous — not only the synthetic videos themselves but also the way they
   promise to blur reality in a country where facts are already up for debate.
   "Deepfakes" have changed the idea that seeing is believing - and could have a huge impact on how future political campaigns unfold.
   (Elyse Samuels/The Washington Post)
   By Editorial Board
   "
   Inside the race to beat 'deepfakes' in 2020
   ' Sections Sign In (
   ) Summer Sale
   Democracy Dies in Darkness
   There's a terrifying trend on the internet
   that could be used to ruin your reputation,
   and no one knows how to stop it
   Lisa Eadicicco, Business Insider
   10.07.2019, 17:01
   Samantha Lee/Business Insider
   The rise of deepfakes, or videos created using AI that can make it
   look like someone said or did something they have never done, has
   FACEBOOK
   ! LINKEDIN
   " TWITTER
   # EMAIL
   $ PRINT
   %
   El Paso Shooting
   This PSA About Fake News From
   Barack Obama Is Not What It
   Appears
   Oscar-winning filmmaker Jordan Peele has a warning for viewers about trusting material they
   encounter online.
   David Mack
   BuzzFeed News Reporter
   Posted on April 17, 2018, at 11:26 a.m. ET
   Sitting before the Stars and Stripes, another flag pinned to his lapel, former president
   Barack Obama appears to be delivering an important message about fake news — but
   something seems slightly...off.
   video-player.buzzfeed.com
   ADVERTISEMENT
   ADVERTISEM
   Tweet Share Copy
   Technology
   Home Video World UK Business Tech Science Stories Entertainment & Arts Health World News TV More
   In fighting deep fakes, mice may be great
   listeners
   By Dave Lee
   North America technology reporter
   Top Stories
   Hong Kong airport cancels flights
   over protests
   2 hours ago
   News Sport Reel Worklife Travel Future Search
   Search the BBC
   Sign in More
   

   View Slide

6. 6
   Andy Polaine – Design in the age of synthetic realities
   “As soon as it works, no
   one calls it AI anymore.”
   - John McCarthy, 1955
   

   View Slide

7. Fake Olds
   7
   Andy Polaine – Design in the age of synthetic realities
   

   View Slide

8. Cover ©1982 National Geographic
   8
   Andy Polaine – Design in the age of synthetic realities
   Image ©Gordan Gahan
   National Geographic, 1982
   

   View Slide

9. 9
   Andy Polaine – Design in the age of synthetic realities
   Photoshop 1.0, 1990
   

   View Slide

10. 10
    Andy Polaine – Design in the age of synthetic realities
    Image: Wikimedia
    Stalin removes Nikolai Yezhov, 1940
    

    View Slide

11. 11
    Andy Polaine – Design in the age of synthetic realities
    Storytelling is the
    most potent
    technology of all
    

    View Slide

12. 12
    Andy Polaine – Design in the age of synthetic realities
    

    View Slide

13. 13
    Andy Polaine – Design in the age of synthetic realities
    A journey through
    synthetic media
    

    View Slide

14. 14
    Andy Polaine – Design in the age of synthetic realities
    Large Scale GAN Training for High Fidelity Natural Image Synthesis. Andrew Brock, Jeff Donahue, Karen Simonyan
    Hi-fidelity synthetic imagery
    Dogball
    

    View Slide

15. 15
    Andy Polaine – Design in the age of synthetic realities
    Synthetic People
    

    View Slide

16. 16
    Andy Polaine – Design in the age of synthetic realities
    Four years of GANs
    Ian Goodfellow: https://twitter.com/goodfellow_ian/status/1084973596236144640
    

    View Slide

17. 17
    Andy Polaine – Design in the age of synthetic realities
    StyleGAN: https://github.com/NVlabs/stylegan
    

    View Slide

18. Dove
    #realmums
    18
    Andy Polaine – Design in the age of synthetic realities
    

    View Slide

19. 19
    Andy Polaine – Design in the age of synthetic realities
    Vid2Vid: https://tcwang0509.github.io/vid2vid/
    

    View Slide

20. 20
    Andy Polaine – Design in the age of synthetic realities
    Vid2Vid: https://tcwang0509.github.io/vid2vid/
    

    View Slide

21. 21
    Andy Polaine – Design in the age of synthetic realities
    Jurassic Park, 1993
    Expression & Pose Editing
    Deep Video Portraits: https://arxiv.org/abs/1805.11714
    

    View Slide

22. 22
    Andy Polaine – Design in the age of synthetic realities
    Few/Single Shot Living Portraits
    Few-Shot Adversarial Learning of Realistic Neural Talking Head Models:
    https://arxiv.org/abs/1905.08233v1
    

    View Slide

23. 23
    Andy Polaine – Design in the age of synthetic realities
    Text to image synthesis
    AttnGAN: https://arxiv.org/abs/1711.10485
    

    View Slide

24. 24
    Andy Polaine – Design in the age of synthetic realities
    Synthetic artists
    

    View Slide

25. 25
    Andy Polaine – Design in the age of synthetic realities
    Jurassic Park, 1993
    ©1993 Universal Pictures | Amblin Entertainment
    

    View Slide

26. 26
    Andy Polaine – Design in the age of synthetic realities
    Rogue One, 2016
    Rogue One
    Deep Fake
    Derpfakes: https://www.youtube.com/watch?v=RiBqZoVe92U
    

    View Slide

27. 27
    Andy Polaine – Design in the age of synthetic realities
    Data Grid
    https://datagrid.co.jp
    

    View Slide

28. 28
    Andy Polaine – Design in the age of synthetic realities
    Willing suspension
    of disbelief
    

    View Slide

29. 29
    Andy Polaine – Design in the age of synthetic realities
    lilmiquela
    Venice Beach
    View Profile
    View More on Instagram
    36,346 likes
    lilmiquela
    *Nicki voice* Let's go to the beach, beach!!! That is all. Have a blessed day bbs
    " # $ %
    ‗
    view all 296 comments
    Add a comment...
    Lil Miquela at the beach. Do synthetic people need sun protection?
    Given the rise of virtual Instagram inhuencer Lil Miquela (who currently has 1.6m followers) and her forays into music,
    advertising and interviewing, it seems likely that the Vrst full GAN-generated artist is around the corner.
    Lil Miquela
    https://www.instagram.com/lilmiquela/
    

    View Slide

30. 30
    Andy Polaine – Design in the age of synthetic realities
    Hatsune Miku
    https://www.instagram.com/hatsune.miku_/
    

    View Slide

31. 31
    Andy Polaine – Design in the age of synthetic realities
    Dalí Lives
    Dalí Museum, Florida: https://www.youtube.com/watch?v=BIDaxl4xqJ4
    

    View Slide

32. 32
    Andy Polaine – Design in the age of synthetic realities
    AI news anchor
    https://twitter.com/XHNews/status/1098221286491271168
    

    View Slide

33. “There is nothing so amazing
    that we can’t get used to it.”
    33
    Andy Polaine – Design in the age of synthetic realities
    Ian McEwan, Machines Like Me
    

    View Slide

34. 34
    Andy Polaine – Design in the age of synthetic realities
    Synthetic media
    and the future of
    design
    

    View Slide

35. Synthesise people, faces, objects, animals
    Capture and re-generate poses and movement
    Modify facial expressions
    Synthesise places and landscapes
    Clone voices
    Generate completely new imagery
    Generate writing
    Effortlessly erase anything from images/video
    Generate GUIs and UX flows
    Link them all together in workflows
    35
    Andy Polaine – Design in the age of synthetic realities
    We can now:
    

    View Slide

36. 36
    Andy Polaine – Design in the age of synthetic realities
    AI-assisted
    design
    

    View Slide

37. 37
    Andy Polaine – Design in the age of synthetic realities
    UIzard
    https://uizard.io
    

    View Slide

38. 38
    Andy Polaine – Design in the age of synthetic realities
    Adobe’s Content-Aware Fill
    

    View Slide

39. 39
    Andy Polaine – Design in the age of synthetic realities
    Designer as
    AI curator
    Image: “squishybrain” on Ganbreeder
    

    View Slide

40. 40
    Andy Polaine – Design in the age of synthetic realities
    Ganbreeder
    http://ganbreeder.app
    

    View Slide

41. 41
    Andy Polaine – Design in the age of synthetic realities
    Runway ML
    http://runwayml.com
    

    View Slide

42. 42
    Andy Polaine – Design in the age of synthetic realities
    Runway ML
    

    View Slide

43. 43
    Andy Polaine – Design in the age of synthetic realities
    Designers’ AI
    assistants
    will become
    valuable IP
    Technology
    Home Video World Asia UK Business Tech Science Stories Entertainment & Arts Health World News T
    1 August 2019
    AI system 'should be recognised as
    inventor'
    By Leo Kelion
    Technology desk editor
    An artificial intelligence system should be recognised as the inventor of two
    ideas in patents filed on its behalf, a team of academics says.
    The AI has designed interlocking food containers that are easy for robots to grasp
    Top Stories
    Amazon fires an 'interna
    crisis', says Macron
    The French president's call fo
    be discussed at the G7 summ
    Brazil’s president.
    2 hours ago
    Cathay Pacific staff spea
    climate of fear
    22 August 2019
    Parents fail to free sons
    daughter's murder
    22 August 2019
    Features
    Why India wants to brea
    decades-old nuclear ple
    Share
    GETTY IMAGES
    News Sport Reel Worklife Travel Future Search
    Search the BBC
    Sign in More
    

    View Slide

44. 44
    Andy Polaine – Design in the age of synthetic realities
    Synthetic Twins
    

    View Slide

45. 30 July 2019
    Chinese vlogger who used filter to look younger caught
    in live-stream glitch
    By Dhruti Shah & Kerry Allen
    BBC News & BBC Monitoring
    Your Highness Qiao Biluo's real appearance (left) and one of her filtered faces (right) from an earlier
    livestream on Douyu
    Fans of a popular Chinese video blogger who called herself "Your Highness
    Latest Posts
    Anti-natalists: The peo
    want you to stop havin
    Who are “anti-natalists” and
    want the gradual extinction
    Twitch CEO apologises
    pornography on Ninja'
    Gundam: The 'nerdy' e
    that led to marriage
    TikTok videos spread c
    change awareness
    Subscribe to our
    BBC Trending
    Share
    DOUYU
    45
    Andy Polaine – Design in the age of synthetic realities
    Synthetic
    Twins
    AI Avatars
    

    View Slide

46. 46
    Andy Polaine – Design in the age of synthetic realities
    Synthetic
    Twins
    Finstas and
    digital
    afterlives
    Image: eterni.me
    

    View Slide

47. 47
    Andy Polaine – Design in the age of synthetic realities
    Synthetic
    Twins
    Privacy &
    surveillance
    avoidance
    Image: URME by Leonardo Selvaggio
    

    View Slide

48. 48
    Andy Polaine – Design in the age of synthetic realities
    Ethical
    challenges
    

    View Slide

49. 49
    Social media ex-removal?
    

    View Slide

50. 50
    Andy Polaine – Design in the age of synthetic realities
    Creative Selection
    StyleGAN: https://github.com/NVlabs/stylegan
    

    View Slide

51. “The whole thing looks very
    natural and no one would
    likely even notice without
    prior knowledge of the
    feature.”
    Andrew O’Hara — AppleInsider
    51
    Image: Fernando Frazão/Agência Brasil
    Andy Polaine – Fjord – Towards a mindful service design – Service Design Days 2018
    Images: Will Sigmon @WSig
    https://twitter.com/WSig/status/1146146914985009154
    Corrected
    Uncorrected
    Apple’s FaceTime Attention Correction
    

    View Slide

52. 52
    Andy Polaine – Design in the age of synthetic realities
    Emotion intervention
    Emotion inference
    Real time emotion inference using CycleGAN: https://www.youtube.com/watch?v=JaQQVrAv9ok
    

    View Slide

53. 53
    Andy Polaine – Design in the age of synthetic realities
    Viewer
    advisory
    labels?
    Warning: emotion filters active
    

    View Slide

54. “You’ve got to rely on people’s ethics.
    You don’t cheat just because the
    technology is available. Machines aren’t
    ethical or unethical — people are.”
    54
    Andy Polaine – Design in the age of synthetic realities
    Brian Steffans, graphics editor at The Los Angeles Times, 1987.
    

    View Slide

55. Thank you.
    Andy Polaine
    @apolaine @fjord
    Newsletter: pln.me/nws
    Slides: pln.me/uxaus-synthetic
    UX Australia, Sydney, 2019
    

    View Slide