Upgrade to Pro — share decks privately, control downloads, hide ads and more …

2016 Current Patents & Inventions

2016 Current Patents & Inventions

This session walks through recent patents filed by Apple and Google to give a glimpse of the possible future of interfaces and technology.

Alexander Meinhardt

March 01, 2016
Tweet

More Decks by Alexander Meinhardt

Other Decks in Technology

Transcript

  1. P A T E N T S & I N

    V E N T I O N S K R U N C H T I M E I NT E RACT IV E
  2. P A T E N T S & I N

    V E N T I O N S Eye tracking
  3. P A T E N T S & I N

    V E N T I O N S Covers a graphical user interface that actively tracks where you're looking, allowing users to control an onscreen input or mouse cursor with just their gaze. Apple / US 8,937,591
  4. P A T E N T S & I N

    V E N T I O N S Apple / US 8,937,591
  5. P A T E N T S & I N

    V E N T I O N S The patent's main focus, however, is on preventing a common eye- tracking issue called the Troxler Effect, an optical illusion which can cause onscreen objects that are stationary to look like they're disappearing. You can see an example in the next slide of a stabilised image and the Troxler Effect in action: Try staring at the cursor in the center and you should notice the purple dots fade away after a few seconds. Troxler Effect
  6. P A T E N T S & I N

    V E N T I O N S Troxler Effect
  7. P A T E N T S & I N

    V E N T I O N S
  8. P A T E N T S & I N

    V E N T I O N S Face recognition for security
  9. P A T E N T S & I N

    V E N T I O N S To solve the issue of having to lock or unlock a device manually, Apple proposes that a product can be configured to lock itself if the user's face is no longer present in front of the camera, and unlock itself when the user's face reappears. Apple / US 2012/0235790
  10. P A T E N T S & I N

    V E N T I O N S Apple / US 2012/0235790
  11. P A T E N T S & I N

    V E N T I O N S Using a forward-facing camera to recognise an individual user, future iPhones and iPads from Apple could automatically customise applications, settings and features to a user's personal preferences once they pick up the device. 
 Each user could customise their personal profile with a unique wallpaper, applications and settings, and that profile would be immediately accessed once the iPad recognises a user's face. Apple / US 2011/0317872
  12. P A T E N T S & I N

    V E N T I O N S Apple / US 2011/0317872
  13. P A T E N T S & I N

    V E N T I O N S Dual lens cameras
  14. P A T E N T S & I N

    V E N T I O N S Apple acquired LinX in April 2015, a camera module maker working on multi-lens systems. Dual lens cameras pack impressive image quality in a smaller size than single aperture cameras, meaning the iPhone 7 Plus may lack a protruding camera lens and be able to take SLR-quality photos — think Canon or Nikon (depth of field). Dual lens cameras
  15. P A T E N T S & I N

    V E N T I O N S Dual lens cameras
  16. P A T E N T S & I N

    V E N T I O N S Dual lens cameras
  17. P A T E N T S & I N

    V E N T I O N S Dual lens cameras
  18. P A T E N T S & I N

    V E N T I O N S Dual lens cameras Noise reduction Images captured by dual lens cameras are brighter and clearer, with significantly reduced noise levels, compared to other smartphone cameras.
  19. P A T E N T S & I N

    V E N T I O N S Dual lens cameras
  20. P A T E N T S & I N

    V E N T I O N S Dual lens cameras 2-3x optical zoom Available detail when zoomed into a photo is also much greater.
  21. P A T E N T S & I N

    V E N T I O N S Dual lens cameras
  22. P A T E N T S & I N

    V E N T I O N S Dual lens cameras Improved Indoor Photos Sensors let in more light, for a photo that is clearer and sharper with less noise.
  23. P A T E N T S & I N

    V E N T I O N S Dual lens cameras
  24. P A T E N T S & I N

    V E N T I O N S Dual lens cameras Improved Low Light Photos The technology is using multiple channels to increase the sensitivity of the camera for better detail. It also keeps exposure times short to cut down on the motion blurring that can impact photo quality in conditions where lighting is not optimal.
  25. P A T E N T S & I N

    V E N T I O N S Dual lens cameras
  26. P A T E N T S & I N

    V E N T I O N S Dual lens cameras 3D depth mapping Useful applications include 3D scanning of objects, sizing of objects, background removal and replacement and gesture recognition. The depth maps also allow for improved refocusing; by knowing the depth at every pixel, the feature allows for a synthetic blur to be applied that emulates a shallow depth.
  27. Dual lens cameras P A T E N T S

    & I N V E N T I O N S
  28. Dual lens cameras Benefits Overview • Depth of field •

    Better color accuracy and uniformity • HDR - higher dynamic range • UHDR - ultra high dynamic range • Low noise levels • Higher resolution • Low module costs • Zero shutter lag • Small size allows for slim devices and edge-to-edge displays
  29. P A T E N T S & I N

    V E N T I O N S 3D depth mapping
  30. P A T E N T S & I N

    V E N T I O N S Covers a technology that will allow hand signals to control a computer app or Apple TV via 3D depth mapping. Apple / US
  31. P A T E N T S & I N

    V E N T I O N S Apple / US
  32. P A T E N T S & I N

    V E N T I O N S Apple / US
  33. P A T E N T S & I N

    V E N T I O N S Covers a 3D sensing device that works with distinct gestures that could control items on a display or TV, such as allowing a user to select an tune on iTunes, or to activate a channel like Netflix without the need of a remote control. Apple / US
  34. P A T E N T S & I N

    V E N T I O N S Apple / US
  35. P A T E N T S & I N

    V E N T I O N S Apple / US
  36. P A T E N T S & I N

    V E N T I O N S Covers user interfaces that are based on three-dimensional sensing, such as Push, Wave and Up Air Gestures. Apple / US 8,933,876
  37. P A T E N T S & I N

    V E N T I O N S Apple / US 8,933,876
  38. P A T E N T S & I N

    V E N T I O N S Covers a method for face detection, which includes capturing a depth map and an image of a scene. One or more locations in the image are selected to test for presence of human faces. Apple / US
  39. P A T E N T S & I N

    V E N T I O N S Apple / US
  40. P A T E N T S & I N

    V E N T I O N S Covers receiving a depth map of a scene containing at least an upper body of a humanoid form. The technology may again serve as a simplified input to application programs, such as responding to users' gestures. Apple / US 9,047,507
  41. P A T E N T S & I N

    V E N T I O N S Apple / US 9,047,507
  42. P A T E N T S & I N

    V E N T I O N S New gestures
  43. P A T E N T S & I N

    V E N T I O N S Covers detection and interpretation of real-world gestures for pinch and zoom, security & more. Apple / US 9,158,454
  44. P A T E N T S & I N

    V E N T I O N S Apple / Knobs with audible feedback
  45. P A T E N T S & I N

    V E N T I O N S Apple / ‚Right-click‘
  46. P A T E N T S & I N

    V E N T I O N S Apple / ‚Okay‘ gesture
  47. P A T E N T S & I N

    V E N T I O N S Google’s Project Soli has invented a new interaction sensor using radar technology that can capture motions of your fingers at up to 10,000 frames per second. Simply put, this technology is so bafflingly accurate that you could operate any device without having to even touch it. Google / Project Soli
  48. P A T E N T S & I N

    V E N T I O N S This chip is actually a miniature gesture radar that captures even the most complex hand movements at close range, at unbelievably hyper speeds and replicates hand gestures. Given the micro size of the chip, it can almost be fitted into literally anything. This technology, if the project is successful, can make the need to touch a device to operate it redundant. Google / Project Soli
  49. P A T E N T S & I N

    V E N T I O N S Google / Project Soli
  50. P A T E N T S & I N

    V E N T I O N S Holograms
  51. P A T E N T S & I N

    V E N T I O N S Covers technology that would allow a device to project holograms. A light emission device would be placed right above the screen, and another thicker material right above that image source. Samsung / December 27, 2015
  52. P A T E N T S & I N

    V E N T I O N S Samsung / December 27, 2015
  53. P A T E N T S & I N

    V E N T I O N S Samsung / December 27, 2015
  54. P A T E N T S & I N

    V E N T I O N S Covers a 3d display system that would mimic a hologram without requiring special glasses. The device would be able to project stereoscopic 3D images to multiple viewers at the same time with complete freedom of movement. Apple / US 7,843,449
  55. P A T E N T S & I N

    V E N T I O N S Covers a device that would be able to direct multiple beams of light through a display to generate an image, while also taking into account the position of the user, who could interact with the 3d holographic images. Apple / US 8,847,919
  56. P A T E N T S & I N

    V E N T I O N S Apple / US 8,847,919
  57. P A T E N T S & I N

    V E N T I O N S Apple / US 8,847,919
  58. P A T E N T S & I N

    V E N T I O N S Apple / US 8,847,919
  59. P A T E N T S & I N

    V E N T I O N S 3D interface effects
  60. P A T E N T S & I N

    V E N T I O N S Covers interaction with a virtual 3D world inside the display and how to prevent the GPU from constantly re-rendering. Apple / US 8,913,056
  61. P A T E N T S & I N

    V E N T I O N S Apple / US 8,913,056
  62. P A T E N T S & I N

    V E N T I O N S Apple / US 8,913,056
  63. P A T E N T S & I N

    V E N T I O N S Automated photo effects
  64. P A T E N T S & I N

    V E N T I O N S Covers photo effects activated by sound, motion and location. Unlike Instagram, users wouldn’t need to select an effect each time they wanted to apply one. Apple / US 8,933,960
  65. P A T E N T S & I N

    V E N T I O N S Person-to-person payments
  66. P A T E N T S & I N

    V E N T I O N S Covers identifying a proximate electronic device of a counter party in a financial transaction, and may provide an encrypted payment packet to the other electronic device. Apple / US 2015/0186887
  67. P A T E N T S & I N

    V E N T I O N S Apple / US 2015/0186887
  68. P A T E N T S & I N

    V E N T I O N S Interactive toys
  69. P A T E N T S & I N

    V E N T I O N S Covers agent interfaces for interactive electronics that support social cues. A doll or toy may be configured to control one or more media devices by detection of a movement or spoken word. Doing so it would „wake up“ and „make eye-contact“. Google / US 2015/0138333
  70. P A T E N T S & I N

    V E N T I O N S Google / US 2015/0138333
  71. P A T E N T S & I N

    V E N T I O N S The future of 3D Touch
  72. P A T E N T S & I N

    V E N T I O N S 3D Touch is a feature which senses how much pressure you apply to the display. The amount of pressure needed to activate the feature can be customized through a Light, Medium, or Firm setting. Where we are
  73. P A T E N T S & I N

    V E N T I O N S Quick actions (Home screen) Peek and Pop (In-app) Where we are Allow apps to deep link and/or perform specific app actions from the home screen. Allow apps to let users preview content and perform related actions within the app, before deciding if they want to view the full content.
  74. P A T E N T S & I N

    V E N T I O N S Shortcut actions Use 3D Touch as an alternative type of “click” (like right-click on a mouse) to let users skip a few steps, or avoid unnecessary steps in a process. Dynamic actions Rely on the amount of pressure applied to signal a variety of actions. Where we are
  75. P A T E N T S & I N

    V E N T I O N S Home screen shortcut actions • Control Center : 3D Touch on WiFi or Bluetooth icon bring up easy options to connect to a network or device • Control Center : Stop the device from changing orientation when watching a video • Calendar: Create a meeting at the last Maps location viewed 1 / Examples
  76. P A T E N T S & I N

    V E N T I O N S 2 / Examples In-app shortcut actions: iOS Mail • Peek into „time reference“ shows a snippet of the calendar, and adds a meeting from there In-app shortcut actions: more • Google search on a highlighted word • Email an image to a favorite contact
  77. P A T E N T S & I N

    V E N T I O N S In-app dynamic actions • Using the amount of pressure applied to a photo to zoom into it • Turn the iPhone into a kitchen scale • 3D Touch for anything that has a “slider”: number of passengers on a flight, radius of a circle in Foursquare 3 / Examples
  78. P A T E N T S & I N

    V E N T I O N S Home screen dynamic actions • 3D Touch as main UI • Set a timer from the Clock app’s home screen icon by using varying levels of pressure • Instead of using the Hue app, change the brightness of its smart lights by applying to the app’s home screen icon • Instead of going into the Nest app, change the temperature from the home screen 4 / Examples
  79. P A T E N T S & I N

    V E N T I O N S Piano rhythm game that uses the pressure of a touch to dictate how loud a note should be. Dynamic actions: Magic Piano
  80. P A T E N T S & I N

    V E N T I O N S A children game that makes letters and words react in an special way depending on how hard you press. Dynamic actions: Endless Alphabet
  81. P A T E N T S & I N

    V E N T I O N S A new generation automotive concept that shows how dynamic actions could be used for changing volume, among other things. Dynamic actions: Synaptics prototype
  82. P A T E N T S & I N

    V E N T I O N S Modular phones
  83. P A T E N T S & I N

    V E N T I O N S The promise: A phone that you can easily upgrade, repair and customise. A modular phone. One universal platform for the whole world. Modular Phones
  84. P A T E N T S & I N

    V E N T I O N S Phonebloks is an open-source modular smartphone concept created and designed by Dave Hakkens, primarily to reduce electronic waste. By attaching individual third-party components to a main board, a user would create a personalised smartphone. In October 2013, Hakkens started collaborating with Motorola and Google on Project Ara. Phonebloks
  85. P A T E N T S & I N

    V E N T I O N S Smartphones based on the Phonebloks system would be sold part by part, as well as in starter sets. Bloks can be replaced at will to replace a broken blok, to upgrade an existing blok, or to expand the functionality of the phone into a specific direction. Bloks would be available in Blokstore, "an app store for hardware", where users could buy new and used bloks as well as sell back their old ones. As of February 2014, the campaign had over 950,000 supporters and a social reach of over 380 million people. Phonebloks
  86. P A T E N T S & I N

    V E N T I O N S ARA is an open hardware platform for creating highly modular smartphones. It includes a structural frame that holds a display, camera or an extra battery. It would allow users to swap out malfunctioning modules or upgrade individual modules, providing longer lifetime cycles and reducing electronic waste. Scheduled to begin pilot testing in the United States in 2016. Google / Project ARA
  87. P A T E N T S & I N

    V E N T I O N S Two frame sizes will be available at first: "mini","medium", and a "large" frame. Each frame is expected to cost around US$15. Modules can provide cameras and speakers, but can also more specialised features, such as medical devices, receipt printers, laser pointers, pico projectors, night vision sensors, or game controller buttons. 
 
 Modules can be hot-swapped without turning the phone off. They will be available both at an official Google store and at third-party stores. Google / Project ARA
  88. P A T E N T S & I N

    V E N T I O N S PuzzlePhone is a modular smartphone, made in Finland. It’s made out of three essential modules, the Heart, the Spine, and the Brain, that can be upgraded, for maximum longevity and performance. Earliest estimated delivery date is September 2016. Puzzlephone
  89. P A T E N T S & I N

    V E N T I O N S RePhone will let you create your own phone in minutes with several modules, an accessible software and a customisable phone case. The open source modular phone kit allows everyone to add cellular communication to different objects. RePhone Kit
  90. P A T E N T S & I N

    V E N T I O N S RePhone is compatible with the Arduino platform, with the Pebble smartwatch and with IFTTT for example to have a link with your pets, toys, motorcycle, robots or drones. 
 It includes a Core Module GSM + BLE, an audio module, a touchscreen, a battery and two precut Kraft Paper to assemble a phone case. This DIY (Do It Yourself) platform lets you create your personal case using a lot of materials, for example wood, plastic, leather, etc. The company has also developed other modules with NFC, GPS, 3G connection, a camera and basic sensors (Accelerometer, Temperature Sensor, Light Sensor). RePhone Kit
  91. P A T E N T S & I N

    V E N T I O N S Fairphone is a Dutch company founded to develop a smartphone which doesn’t contain conflict minerals (gold, tin, tantalum and tungsten) and using fair labor conditions for the workforce along all the supply chain. Fairphone 1 sold 60.000 units, Fairphone 2 was released in 2015. Fairphone
  92. P A T E N T S & I N

    V E N T I O N S Fairphone 2 is an innovative smartphone in all its aspects. This smartphone introduces a new and innovative modular construction technology. Thanks to this architecture, anyone can do basic maintenance on his or her phone without external support. It runs Android 5.1 and a lot of high quality components. Fairphone
  93. P A T E N T S & I N

    V E N T I O N S The company has developed different types of modules: A Core unit (memory, flash storage, modem, antennas, the SIMs and MicroSD card and motion sensors), a display unit with the LCD screen and the receiver module (receiver, headset connector, front-facing camera, noise-cancelling microphone, ambient light and proximity sensor and the notification LED). It also hosts a rear camera module and a speaker unit, which includes also the vibration mechanism, main microphone and a USB connector. The various modules may be replaced with other components with better features, for example, an IR camera instead of the normal rear-camera unit. Fairphone
  94. P A T E N T S & I N

    V E N T I O N S The company wants to raise awareness about the issues surrounding mining and existing alternatives, in particular, improve working conditions for miners, introduce local wages, eliminate child labor and reduce environmental degradation caused by mining practices. Other goals are to establish collaborative, mutually beneficial, and transparent relationships with manufacturers to ensure worker representation, safe working conditions and fair pay. Fairphone
  95. P A T E N T S & I N

    V E N T I O N S The G5 is a next-gen smartphone that introduces two modular add-ons. It has an accessory slot at the bottom, which allows users to remove the battery and to swap in different modules: LG CAM Plus, LG Hi-Fi Plus” Bang & Olufsen DAC accessory, etc. LG is supposed to launch G5 in April. LG G5
  96. P A T E N T S & I N

    V E N T I O N S Beyond the smartphone
  97. P A T E N T S & I N

    V E N T I O N S Virtual & Augmented Reality
  98. P A T E N T S & I N

    V E N T I O N S 2016 will be the year of Virtual Reality, with companies like Samsung with the Gear VR giveaway and Facebook with Social VR and partnership with Samsung investing hard on it. Virtual Reality
  99. P A T E N T S & I N

    V E N T I O N S Head-mounted displays market will progress toward mainstream adoption for consumers and enterprise use. New virtual reality products for consumers, such as the HTC Vive, Oculus Rift, Sony PlayStation VR, and Microsoft HoloLens are expected to be available along with video games and entertainment content as well as business applications. Film producers and sports leagues will augment their traditional content through HMDs to enhance their customer experiences by creating interactive attractions, movies, and sporting events that make the content more personal and meaningful. Virtual Reality
  100. Virtual Reality Samsung Gear VR 
 Free / 99 €

    HTC Vive
 Pre-orders 799 $ Oculus Rift
 
 July 2016 - 599 $ Sony Playstation VR
 October 2016 - 400 $
  101. Virtual Reality *Statista 2016 Samsung Gear VR Oculus Rift HTC

    Vive Sony Playstation VR Others Sold pieces in Mio. 5,0 Mio. 3,6 Mio. 2,1 Mio. 1,4 Mio. 0,1 Mio. VR Headsets unit sales forecast Worldwide in 2016
  102. P A T E N T S & I N

    V E N T I O N S Apple had been creating a VR team and recent revelations indicate that Apple is now accelerating their VR project. There are patents that go back to iPod times. Apple has built a large team that is experimenting with headsets and other technologies. Hundreds of employees are part of a "secret research unit" exploring AR and VR, building prototypes of virtual reality headsets that are similar to the Oculus Rift and the Hololens from Microsoft. Multiple prototypes of "possible headset configurations" have been created in recent months, with Apple's interest reportedly inspired by the Oculus Rift. Virtual Reality
  103. P A T E N T S & I N

    V E N T I O N S Virtual Reality
  104. P A T E N T S & I N

    V E N T I O N S Apple has filed a patent related a device / glasses that projects content, being aware of objects in its field of view, recognising such objects as suitable for projection of content thereon. For instance, like we see in the picture, the interface floating that lets you buy a book, when looking at the physical book. The projection device may adapt to the geometry and character of the objects by controlling scale, distortion, focus of the projected content, and varying the projected content itself. Augmented Reality
  105. P A T E N T S & I N

    V E N T I O N S Augmented Reality
  106. P A T E N T S & I N

    V E N T I O N S HoloLens is a "mixed reality" computing platform, enabling applications in which the live presentation of physical real-world elements is incorporated with that of virtual elements. Microsoft HoloLens
  107. P A T E N T S & I N

    V E N T I O N S Consists of a smart-glasses headset (visor) that is a cordless, self-contained Windows 10 computer.
 
 Uses advanced sensors and spatial sound with a natural user interface that the user interacts with through gaze, voice, and hand gestures. Microsoft HoloLens
  108. P A T E N T S & I N

    V E N T I O N S Microsoft HoloLens
  109. P A T E N T S & I N

    V E N T I O N S Microsoft HoloLens
  110. P A T E N T S & I N

    V E N T I O N S Microsoft HoloLens
  111. P A T E N T S & I N

    V E N T I O N S Microsoft HoloLens
  112. P A T E N T S & I N

    V E N T I O N S Microsoft HoloLens
  113. P A T E N T S & I N

    V E N T I O N S Magic Leap is a US startup that is working on a head-mounted virtual retinal display.
 Hardware not presented to public yet.
 ”A Google Glass on steroids that can seamlessly blend computer- generated graphics with the real world“. Magic Leap
  114. P A T E N T S & I N

    V E N T I O N S Works by projecting light on your retina.
 Your eyes can move around and focus on things, unlike using VR.
 Aiming to be worn all day long in public while being social.
 Promises to make virtual experiences happen more naturally. Magic Leap
  115. P A T E N T S & I N

    V E N T I O N S „Dreams of being a full computer“.
 Will run its own OS.
 Comparable but arguably better than Microsoft HoloLens. Magic Leap
  116. P A T E N T S & I N

    V E N T I O N S Magic Leap Announces $793.5 million in New Funding Dania Beach, Florida, February 2, 2016 - Magic Leap, Inc. announced $793.5 million in new funding today, led by Alibaba Group. Continued investment comes from Google Inc., Qualcomm Incorporated, through its venture investment group, Qualcomm Ventures, and others. Magic Leap
  117. P A T E N T S & I N

    V E N T I O N S Emergency Free Fall Orientation
  118. P A T E N T S & I N

    V E N T I O N S Covers the characteristic of the gyroscopic torque to be used to reorient the mobile device during the free fall. Apple / US 2016/0018223 If a device is oriented during free fall such that it will likely fall directly onto its display, the characteristic of the gyroscopic torque may be selected so that the device will instead impact on its edge or side.
  119. P A T E N T S & I N

    V E N T I O N S Apple / US 2016/0018223
  120. Q U E S T I O N S ?

    R X U F O U M
  121. Alexander Meinhardt krunchtime.org KRUN C H T I M E

    I NT E R ACT I V E Iker Ortiz de Arri COAU T H ORS