Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Design in the Age of Synthetic Realities

Andy Polaine
August 29, 2019

Design in the Age of Synthetic Realities

Presentation at UX Australia 2019

AI-generated and mixed realities are blurring the boundaries of “truth” and challenging how we value it. Synthetic Realities have reached new heights of sophistication, sparking controversy, and also fascination, about its creative possibilities.

Customers will increasingly expect brands to meet them halfway to supplement the realities they desire. Next, they will expect reality to adapt to them in real time without any conscious request. And the role of designers will be to set the stage on which these experiences happen, often curating AI systems rather than designing themselves.

UX practitioners face the double task of designing the interfaces to these new products, tools and services as well as seeing design for digital disrupted as radically as the invention of Photoshop and desktop publishing did three decades ago. If anyone can summon up an entirely generated image or an interface just by speaking its description, what is the future role of the designer?

The mixture of fear and fascination of the power of synthetic media is understandable but wrongly frames its future trajectory. This talk takes a view on synthetic realities not as something brand new, but the latest iteration of a long history of re-defining truth and reality through media manipulation.

Using visually compelling examples from the latest research and work in synthetic and generated media and charting their convergence, we'll see just how quickly they will be absorbed by culture, radically shake-up the fields of design, advertising and marketing in a very short time-frame, and transform the role of future designers and creatives. It raises important questions of design, ethics, and craft that the UX and broader design community will have to find answers to pretty damn quickly.

Andy Polaine

August 29, 2019
Tweet

More Decks by Andy Polaine

Other Decks in Design

Transcript

  1. Design in the age of
    synthetic realities
    Design and Innovation from
    Accenture Interactive
    Andy Polaine
    Group Design Director – Client Evolution
    @apolaine | @fjord
    Slides: pln.me/uxaus-synthetic
    UX Australia, Sydney, 2019

    View full-size slide

  2. 2
    Andy Polaine – Design in the age of synthetic realities
    Synthetic realities
    trends.fjordnet.com

    View full-size slide

  3. 3
    Andy Polaine – Design in the age of synthetic realities
    Image source: Adrian on Unsplash
    Jordan Peele’s Obama PSA

    View full-size slide

  4. 4
    Andy Polaine – Design in the age of synthetic realities
    Deep Fakes

    View full-size slide

  5. 5
    Andy Polaine – Design in the age of synthetic realities
    Bundesweite Umfrage
    Jetzt Umfrage beantworten und exklusives Dankeschön-Paket sichern
    MDM ÖFFNEN
    The Post's View
    Deepfakes are dangerous — and they target a huge
    weakness
    Opinion

    June 16
    DEEPFAKES ARE dangerous — not only the synthetic videos themselves but also the way they
    promise to blur reality in a country where facts are already up for debate.
    "Deepfakes" have changed the idea that seeing is believing - and could have a huge impact on how future political campaigns unfold.
    (Elyse Samuels/The Washington Post)
    By Editorial Board
    "
    Inside the race to beat 'deepfakes' in 2020
    ' Sections Sign In (
    ) Summer Sale
    Democracy Dies in Darkness
    There's a terrifying trend on the internet
    that could be used to ruin your reputation,
    and no one knows how to stop it
    Lisa Eadicicco, Business Insider
    10.07.2019, 17:01
    Samantha Lee/Business Insider
    The rise of deepfakes, or videos created using AI that can make it
    look like someone said or did something they have never done, has
    FACEBOOK
    ! LINKEDIN
    " TWITTER
    # EMAIL
    $ PRINT
    %
    El Paso Shooting
    This PSA About Fake News From
    Barack Obama Is Not What It
    Appears
    Oscar-winning filmmaker Jordan Peele has a warning for viewers about trusting material they
    encounter online.
    David Mack
    BuzzFeed News Reporter
    Posted on April 17, 2018, at 11:26 a.m. ET
    Sitting before the Stars and Stripes, another flag pinned to his lapel, former president
    Barack Obama appears to be delivering an important message about fake news — but
    something seems slightly...off.
    video-player.buzzfeed.com
    ADVERTISEMENT
    ADVERTISEM
    Tweet Share Copy
    Technology
    Home Video World UK Business Tech Science Stories Entertainment & Arts Health World News TV More
    In fighting deep fakes, mice may be great
    listeners
    By Dave Lee
    North America technology reporter
    Top Stories
    Hong Kong airport cancels flights
    over protests
    2 hours ago
    News Sport Reel Worklife Travel Future Search
    Search the BBC
    Sign in More

    View full-size slide

  6. 6
    Andy Polaine – Design in the age of synthetic realities
    “As soon as it works, no
    one calls it AI anymore.”
    - John McCarthy, 1955

    View full-size slide

  7. Fake Olds
    7
    Andy Polaine – Design in the age of synthetic realities

    View full-size slide

  8. Cover ©1982 National Geographic
    8
    Andy Polaine – Design in the age of synthetic realities
    Image ©Gordan Gahan
    National Geographic, 1982

    View full-size slide

  9. 9
    Andy Polaine – Design in the age of synthetic realities
    Photoshop 1.0, 1990

    View full-size slide

  10. 10
    Andy Polaine – Design in the age of synthetic realities
    Image: Wikimedia
    Stalin removes Nikolai Yezhov, 1940

    View full-size slide

  11. 11
    Andy Polaine – Design in the age of synthetic realities
    Storytelling is the
    most potent
    technology of all

    View full-size slide

  12. 12
    Andy Polaine – Design in the age of synthetic realities

    View full-size slide

  13. 13
    Andy Polaine – Design in the age of synthetic realities
    A journey through
    synthetic media

    View full-size slide

  14. 14
    Andy Polaine – Design in the age of synthetic realities
    Large Scale GAN Training for High Fidelity Natural Image Synthesis. Andrew Brock, Jeff Donahue, Karen Simonyan
    Hi-fidelity synthetic imagery
    Dogball

    View full-size slide

  15. 15
    Andy Polaine – Design in the age of synthetic realities
    Synthetic People

    View full-size slide

  16. 16
    Andy Polaine – Design in the age of synthetic realities
    Four years of GANs
    Ian Goodfellow: https://twitter.com/goodfellow_ian/status/1084973596236144640

    View full-size slide

  17. 17
    Andy Polaine – Design in the age of synthetic realities
    StyleGAN: https://github.com/NVlabs/stylegan

    View full-size slide

  18. Dove
    #realmums
    18
    Andy Polaine – Design in the age of synthetic realities

    View full-size slide

  19. 19
    Andy Polaine – Design in the age of synthetic realities
    Vid2Vid: https://tcwang0509.github.io/vid2vid/

    View full-size slide

  20. 20
    Andy Polaine – Design in the age of synthetic realities
    Vid2Vid: https://tcwang0509.github.io/vid2vid/

    View full-size slide

  21. 21
    Andy Polaine – Design in the age of synthetic realities
    Jurassic Park, 1993
    Expression & Pose Editing
    Deep Video Portraits: https://arxiv.org/abs/1805.11714

    View full-size slide

  22. 22
    Andy Polaine – Design in the age of synthetic realities
    Few/Single Shot Living Portraits
    Few-Shot Adversarial Learning of Realistic Neural Talking Head Models:
    https://arxiv.org/abs/1905.08233v1

    View full-size slide

  23. 23
    Andy Polaine – Design in the age of synthetic realities
    Text to image synthesis
    AttnGAN: https://arxiv.org/abs/1711.10485

    View full-size slide

  24. 24
    Andy Polaine – Design in the age of synthetic realities
    Synthetic artists

    View full-size slide

  25. 25
    Andy Polaine – Design in the age of synthetic realities
    Jurassic Park, 1993
    ©1993 Universal Pictures | Amblin Entertainment

    View full-size slide

  26. 26
    Andy Polaine – Design in the age of synthetic realities
    Rogue One, 2016
    Rogue One
    Deep Fake
    Derpfakes: https://www.youtube.com/watch?v=RiBqZoVe92U

    View full-size slide

  27. 27
    Andy Polaine – Design in the age of synthetic realities
    Data Grid
    https://datagrid.co.jp

    View full-size slide

  28. 28
    Andy Polaine – Design in the age of synthetic realities
    Willing suspension
    of disbelief

    View full-size slide

  29. 29
    Andy Polaine – Design in the age of synthetic realities
    lilmiquela
    Venice Beach
    View Profile
    View More on Instagram
    36,346 likes
    lilmiquela
    *Nicki voice* Let's go to the beach, beach!!! That is all. Have a blessed day bbs
    " # $ %

    view all 296 comments
    Add a comment...
    Lil Miquela at the beach. Do synthetic people need sun protection?
    Given the rise of virtual Instagram inhuencer Lil Miquela (who currently has 1.6m followers) and her forays into music,
    advertising and interviewing, it seems likely that the Vrst full GAN-generated artist is around the corner.
    Lil Miquela
    https://www.instagram.com/lilmiquela/

    View full-size slide

  30. 30
    Andy Polaine – Design in the age of synthetic realities
    Hatsune Miku
    https://www.instagram.com/hatsune.miku_/

    View full-size slide

  31. 31
    Andy Polaine – Design in the age of synthetic realities
    Dalí Lives
    Dalí Museum, Florida: https://www.youtube.com/watch?v=BIDaxl4xqJ4

    View full-size slide

  32. 32
    Andy Polaine – Design in the age of synthetic realities
    AI news anchor
    https://twitter.com/XHNews/status/1098221286491271168

    View full-size slide

  33. “There is nothing so amazing
    that we can’t get used to it.”
    33
    Andy Polaine – Design in the age of synthetic realities
    Ian McEwan, Machines Like Me

    View full-size slide

  34. 34
    Andy Polaine – Design in the age of synthetic realities
    Synthetic media
    and the future of
    design

    View full-size slide

  35. Synthesise people, faces, objects, animals
    Capture and re-generate poses and movement
    Modify facial expressions
    Synthesise places and landscapes
    Clone voices
    Generate completely new imagery
    Generate writing
    Effortlessly erase anything from images/video
    Generate GUIs and UX flows
    Link them all together in workflows
    35
    Andy Polaine – Design in the age of synthetic realities
    We can now:

    View full-size slide

  36. 36
    Andy Polaine – Design in the age of synthetic realities
    AI-assisted
    design

    View full-size slide

  37. 37
    Andy Polaine – Design in the age of synthetic realities
    UIzard
    https://uizard.io

    View full-size slide

  38. 38
    Andy Polaine – Design in the age of synthetic realities
    Adobe’s Content-Aware Fill

    View full-size slide

  39. 39
    Andy Polaine – Design in the age of synthetic realities
    Designer as
    AI curator
    Image: “squishybrain” on Ganbreeder

    View full-size slide

  40. 40
    Andy Polaine – Design in the age of synthetic realities
    Ganbreeder
    http://ganbreeder.app

    View full-size slide

  41. 41
    Andy Polaine – Design in the age of synthetic realities
    Runway ML
    http://runwayml.com

    View full-size slide

  42. 42
    Andy Polaine – Design in the age of synthetic realities
    Runway ML

    View full-size slide

  43. 43
    Andy Polaine – Design in the age of synthetic realities
    Designers’ AI
    assistants
    will become
    valuable IP
    Technology
    Home Video World Asia UK Business Tech Science Stories Entertainment & Arts Health World News T
    1 August 2019
    AI system 'should be recognised as
    inventor'
    By Leo Kelion
    Technology desk editor
    An artificial intelligence system should be recognised as the inventor of two
    ideas in patents filed on its behalf, a team of academics says.
    The AI has designed interlocking food containers that are easy for robots to grasp
    Top Stories
    Amazon fires an 'interna
    crisis', says Macron
    The French president's call fo
    be discussed at the G7 summ
    Brazil’s president.
    2 hours ago
    Cathay Pacific staff spea
    climate of fear
    22 August 2019
    Parents fail to free sons
    daughter's murder
    22 August 2019
    Features
    Why India wants to brea
    decades-old nuclear ple
    Share
    GETTY IMAGES
    News Sport Reel Worklife Travel Future Search
    Search the BBC
    Sign in More

    View full-size slide

  44. 44
    Andy Polaine – Design in the age of synthetic realities
    Synthetic Twins

    View full-size slide

  45. 30 July 2019
    Chinese vlogger who used filter to look younger caught
    in live-stream glitch
    By Dhruti Shah & Kerry Allen
    BBC News & BBC Monitoring
    Your Highness Qiao Biluo's real appearance (left) and one of her filtered faces (right) from an earlier
    livestream on Douyu
    Fans of a popular Chinese video blogger who called herself "Your Highness
    Latest Posts
    Anti-natalists: The peo
    want you to stop havin
    Who are “anti-natalists” and
    want the gradual extinction
    Twitch CEO apologises
    pornography on Ninja'
    Gundam: The 'nerdy' e
    that led to marriage
    TikTok videos spread c
    change awareness
    Subscribe to our
    BBC Trending
    Share
    DOUYU
    45
    Andy Polaine – Design in the age of synthetic realities
    Synthetic
    Twins
    AI Avatars

    View full-size slide

  46. 46
    Andy Polaine – Design in the age of synthetic realities
    Synthetic
    Twins
    Finstas and
    digital
    afterlives
    Image: eterni.me

    View full-size slide

  47. 47
    Andy Polaine – Design in the age of synthetic realities
    Synthetic
    Twins
    Privacy &
    surveillance
    avoidance
    Image: URME by Leonardo Selvaggio

    View full-size slide

  48. 48
    Andy Polaine – Design in the age of synthetic realities
    Ethical
    challenges

    View full-size slide

  49. 49
    Social media ex-removal?

    View full-size slide

  50. 50
    Andy Polaine – Design in the age of synthetic realities
    Creative Selection
    StyleGAN: https://github.com/NVlabs/stylegan

    View full-size slide

  51. “The whole thing looks very
    natural and no one would
    likely even notice without
    prior knowledge of the
    feature.”
    Andrew O’Hara — AppleInsider
    51
    Image: Fernando Frazão/Agência Brasil
    Andy Polaine – Fjord – Towards a mindful service design – Service Design Days 2018
    Images: Will Sigmon @WSig
    https://twitter.com/WSig/status/1146146914985009154
    Corrected
    Uncorrected
    Apple’s FaceTime Attention Correction

    View full-size slide

  52. 52
    Andy Polaine – Design in the age of synthetic realities
    Emotion intervention
    Emotion inference
    Real time emotion inference using CycleGAN: https://www.youtube.com/watch?v=JaQQVrAv9ok

    View full-size slide

  53. 53
    Andy Polaine – Design in the age of synthetic realities
    Viewer
    advisory
    labels?
    Warning: emotion filters active

    View full-size slide

  54. “You’ve got to rely on people’s ethics.
    You don’t cheat just because the
    technology is available. Machines aren’t
    ethical or unethical — people are.”
    54
    Andy Polaine – Design in the age of synthetic realities
    Brian Steffans, graphics editor at The Los Angeles Times, 1987.

    View full-size slide

  55. Thank you.
    Andy Polaine
    @apolaine @fjord
    Newsletter: pln.me/nws
    Slides: pln.me/uxaus-synthetic
    UX Australia, Sydney, 2019

    View full-size slide