rough slide Sunday, 10 April 2011 We’re a digital service design agency. That means we design user experiences for services that are used on more than one platform - often mobile, tablets like the iPad, digital TV as well as web.
project across 16 partners & 7 countries Investigating user-centred design of interconnected embedded objects Fjord’s role covers developing design, concepting and research methods, running trials We’re currently at the stage of identifying challenges and generating initial concepts Sunday, 10 April 2011 All of our client work to date has been around designing for things with screens. But we’re now working on an EU funded research project that has given us the opportunity to think beyond the screen. Although we are working with the above partners, this presentation represents Fjord’s view. The rest of this talk is a quick tour of some of the issues we’ve been thinking about on this project. We’re at an early stage and this is a massive area of ongoing research, but we are starting to propose some challenges and suggestions as to how we might overcome them
focus of attention rather than allowing it to fade into the background.” Mark Weiser Sunday, 10 April 2011 In 1991, Mark Weiser (the ‘father of ubiquitous computing’) said...
solely with slick, glowing rectangles” Timo Arnall image - The Onion Sunday, 10 April 2011 ...20 years later, we’ve not really solved this, have we? Our day to day lives involve many interactions with objects, but most of our interactions with computing still happen through the abstracted world of what Timo Arnall calls ‘slick, glowing rectangles’.
However, we’re starting to see digital dissolve more into the physical world. Of course, people like Weiser have been talking about this for a long time now. But things are now starting to happen in the mainstream, here and now, which pose new challenges for UX. We think that in the next couple of years, UX designers are going to have the opportunity to design things that involve not just screens, but services and physical objects for the world around them.
going to be like this... Sunday, 10 April 2011 A lot of the classic ubicomp (ubiquitous computing) research has been focused on trying to understand the ‘bigger picture’ such as smart environments and homes. There’s been lots of promise in this but very few good examples have made it out of research and into the mainstream.
like this... rough slide When your shipment is vital, know it’s vital signs. Sunday, 10 April 2011 ‘The internet of things’: everyday objects becoming interconnected and increasingly able to sense their environment. Currently 98% of all processors are in embedded devices, not multi-purpose computers.... and they are becoming increasingly connected. An example of this is Fedex Senseaware: sensors track the location, temperature of packages and whether they have been opened. For example, if temperature sensitive medical supplies are in danger of being destroyed en route, the package can be returned and a new one sent out, saving vital time.
Sunday, 10 April 2011 Everyday objects getting smarter This is RTMM: real time medication monitoring - made by Evalan, one of our partners on Smarcos... ...a smart pill bottle that knows when the user has taken their pills and can share this data with the user’s doctor or family. It can also remind the user to take them, via SMS or phone call However many of these smart objects are still closed and proprietary systems.
of other things are coming online... Sunday, 10 April 2011 Screen based media aren’t going away. But lots of other types of object are becoming connected and will be forming part of digital services and the experiences we design. Some of them will be objects we interact with directly. They might take lots of forms, and some of them won’t have screens. Some of these objects will just talk to other objects, but will provide data that’s used by things we do interact with. Real (and perhaps crazy) examples of these that someone somewhere has made include dog tags that tweet, connected umbrellas, coffee machines smart home heating and lighting systems. We’re still experimenting with what can be done here and some of the examples might seem flippant. The point is that objects that were not connected can now be connected, and even if we’re not designing those objects, they can have an impact on what we do. For example, right now, you might design websites that sell washing machines. Sooner or later, you may have sensor data from those machines that enables much better after sales service.. how does that change the service you provide? Or you may work for an environmental organisation. What difference would widespread live pollution data make to you? These are examples of ways in which connected devices enable more complex services, which will make designing those services more complex, and, we would say, more exciting. Either way, IA has a lot to offer in terms of both top down experience design and bottom up information sensemaking.
2010 | Confidential New Verizon and AT&T subscriptions May - August 2010 Sunday, 10 April 2011 By way of evidence... As of February 2011, both these networks now have more machine subscribers than human subscribers. That’s currently things like e readers, vehicle tracking systems and alarm monitoring services. http://www.economist.com/blogs/babbage/2010/08/internet_things
| Confidential IMS Research, Ericsson ... or 50 billion? Sunday, 10 April 2011 ... that’s TVs, cars, mobiles and lots and lots of other random things. This is actually a conservative estimate - Ericsson predict 50 billion connected devices.
Over the next couple of years, this stuff is due to hit the mainstream and will affect the work UX designers do on an increasing basis Here’s what we think this might mean for design...
a physical form Sunday, 10 April 2011 We can give digital services a physical embodiment. A simple example is London’s Oyster card - a passive RFID chip embedded in a plastic card, linked up to a service which allows you to charge it up with credit. The card itself is just an identifier, it has no intrinsic value. You wave it over readers and credit is deducted from your account, which lives remotely in a database. But in users’ minds, the card *is* the service.
a digital form Sunday, 10 April 2011 Another way to approach design is enhancing existing products with digital services. “Webkinz are toy stuffed animals that were originally released by the Canadian Ganz company on April 29, 2005. The toys are similar to many other small plush toys, however, each Webkinz toy has an attached tag with a unique "Secret Code" printed on it that allows access to the "Webkinz World" website. On Webkinz World, the Secret Code allows the user to own a virtual versionof the pet for online play for a limited time (one year).” - http://en.wikipedia.org/wiki/Webkinz
text styles Live data enables smarter services. Sunday, 10 April 2011 We design for and with live data to enrich our services. “SFpark works by collecting and distributing real-time information about where parking is available so drivers can quickly find open spaces. To help achieve the right level of parking availability, SFpark will periodically adjust meter pricing up and down to match demand. Demand-responsive pricing encourages drivers to park in underused areas and garages, reducing demand in overused areas. With SFpark, real-time data and demand-responsive pricing work together to readjust parking patterns in the City so that parking is easier to find.” http://sfpark.org/
few of the key challenges we think UX designers will have to be prepared for, and some suggested ways to do things differently. We’re just working this stuff out ourselves... these are some of the issues we hope to be able to research over the next couple of years. These touch on bigger issues - they’re important for this but each is a huge topic in its own right
April 2011 Classic usability tends to focus on one user using one device and one service to do one task at a time. That’s increasingly not what’s actually going on. Our relationship with devices and services is getting more complicated. It’s really important to think not just about device, but service design: how your user experience works across multiple devices. The device is no longer the unit of experience... the service is. As embedded components come online, digital services will have to cope with increasing complexity in several ways...
image: Vivotech.com Sunday, 10 April 2011 Some objects will be touchpoints or data sources for multiple services. Mobiles will play a huge role in this. - Apps are a way to access different services - NFC, beginning to appear in phones, enables multiple types of payment service. One interaction could take money from your credit card, bank account, Paypal... how do you make sure it’s the one you intended to use?
Fjord 2010 | Confidential Sunday, 10 April 2011 But there will be others which may not have screens to show us what they are up to. For example, in New Songdo, a Korean ubiquitous city in development, there is a student smartcard that can be used to get into school, pay for food, as a library card, and as a bus pass (which can be used to notify students’ parents that the student is on the bus and expected home soon- no more secretly sloping off to hang out with friends after school). Although it’s a simple object, and not even terribly smart, it’s a good example of an object with dynamic functionality, dependent on context. Users have to use context of use to infer what it’s doing, and what information it is exchanging, every time it’s swiped.
Confidential Sunday, 10 April 2011 Individual services are being delivered across multiple devices Service coherence is a key part of UX- the service being in sync between multiple devices. Facebook notifications are a current example of this working badly. BBC used to talk about PC web as second screen for iPlayer, then mobile as third screen, but is now talking about creating orchestrated media - a synchronised iPlayer experience across all screens at once. This would allow users to start an interaction on one device and continue it on another without losing their place in the task, and deliver different and complementary experiences across different devices.
Confidential Sunday, 10 April 2011 Non screened device example... Ford Sync: an in car connectivity system that enables the car to be used as a controller for the phone, so users can control the phone’s music player with the car’s voice recognition system, and more. http://www.ford.com/technology/sync/about/
For a while now, we’ve been seeing a trend for services to integrate with other services for the bits those services already do well, such as 1Password, which integrates with Dropbox for synchronisation across devices.
Services can be used to integrate objects, too. Withings, the WiFi body scale shown in our video, shares data with a variety of different fitness and weight loss services, such as RunKeeper, Lose It!, Daily Burn. http://www.withings.com/en/bodyscale/sharing
is being produced in both the physical and digital space, and can be shared in near real time. How do we as designers leverage this huge amount of increasing complex data to help enrich the services we design, and aid us in designing new forms of services?
2011 The traditional web 2.0 way of leveraging this data would be that of aggregation. Example: Flipboard, which aggregate various web feeds such as news sites RSS, Twitter, Facebook etc into a magazine format.
Text Sunday, 10 April 2011 This aggregation is also happening on an organic level, as in the example of the pdxboom hashtag. Just after 8 PM on Sunday, March 28th 2010, a deafening BOOM was heard throughout SE Portland. Quickly the twitter community had adopted the hash tag #pdxboom. In less than 30mins there was a google map displaying where individuals heard the bang and how loud it was. This quickly began to show a pattern and allowed the police to establish the source and cause (it was a pipebomb). Granted it wasn’t the most scientific approach and there are other established platforms such as Ushahidi that would have been better suited. Instead a new platform was created around this individual event in almost real time.
Text Sunday, 10 April 2011 The previous example shows these platforms being formed by the community/users but they can also be created out of leveraging existing data beyond its intended purpose. This was shown recently after the earthquake and tsunami in Japan, Honda opened up their traffic data service and plotted the data onto a public Google map. This information showed in near real time the conditions of the roads and which roads were passable and which ones were not. This is a service that you wouldn’t have expected from Honda as it leveraged a Honda asset beyond its intended purpose. --- http://www.thetruthaboutcars.com/2011/03/honda-gives-devastated-japan-something-badly-needed-information/
April 2011 We can also leverage hardware beyond its intended use, as it becomes more connected and its data more accessible. If you were designing a wireless security system for a home, the same sensors used for the security system could be used as a health system for the elderly e.g pick up on if the person has fallen over or has an unusual behaviour/movement pattern (key in diagnosing the early signs of dementia). do we need to start thinking about designing ‘apps for you house’?
of hardware http://www.flickr.com/photos/nolifebeforecoffee/ Sunday, 10 April 2011 We can agree to hardware for one reason but it can be used in other ways. “ in Wellington, for example, we were told that the surveillance cameras that voters approved to help manage traffic congestion had been repurposed for crime prevention, without a corresponding degree of public consultation. Let the image stream coming off of them be provided with a facial-recognition algorithm, and you’ve got an entirely different kind of system on your hands, with entirely different potentials and vastly expanded implications. Yet the cameras, domed or otherwise, look no different from one day to the next.” Adam Greenfield As both designers and users we need to be aware of how data is and also could be used in the future, and ensure that the right safeguards are in place and their is transparency in the service of how data is collected and used. http://speedbird.wordpress.com/2010/04/28/neopanoptical/ http://www.flickr.com/photos/nolifebeforecoffee/
Victor Rozek Sunday, 10 April 2011 Privacy management is much more than a bunch of tick boxes and security settings. It incorporates less tangible elements such as 'appropriate use' and a user’s ‘comfort level’ etc. Many people already find managing privacy too difficult on Facebook and share things with people they didn’t mean to share them with. It’s going to get a lot more complicated.
Sunday, 10 April 2011 Passive vs active - person to service can be active.|| service to service is going to be passive || Google Latitude didn’t take off partly because users didn’t always want passively to share their location. On Foursquare, which requires active check in, you can simply choose not to check in. But active controls won’t scale up to really complex services or services that talk to each other. We can design individual services that use a user’s data ‘appropriately’. However, gaps between individual services can form where personal data may be leaked. With more objects becoming connected and sharing what’s deemed 'appropriate' data dependent on their service, the potential for gaps that can leak personal data increases, especially when data we share with one service ends up being used by another service. How do you know where your data is going to end up? How can service designers design for transparency - so that users can see when their data is being shared, and where it will end up?
get creepier http://ilektrojohn.github.com/creepy/ Sunday, 10 April 2011 “Creepy is an application that allows you to gather geolocation related information about users from social networking platforms and image hosting services. The information is presented in a map inside the application where all the retrieved data is shown accompanied with relevant information (i.e. what was posted from that specific location) to provide context to the presentation.” Information that seemed innocent when shared can be aggregated to build up a scarily detailed picture of someone’s daily life. Will we see the emergence of privacy brokers to help us manage our data in the face of all this complexity? If brokers do emerge, what kind of organisations would we trust enough to manage this? Some have suggested the mobile operators... Also, what degree of control would we be happy to hand over to them?
the risks Sunday, 10 April 2011 ...there’s huge potential in all the data that can be gathered and mined... to deliver new types of service and create better user experiences. ... but we need to do more work to understand how to help people manage this.
talk about ‘embodied cognition’*: the idea that the way we think is shaped by, and inseparable from, our physical experiences of interacting with the world. (Dourish, McCullough) For example, the idea that up is good and down is bad is rooted in your physical experience of living with gravity. “I’m feeling down today” is bad. Up (and fast) is good... “I’m feeling upbeat”. We say someone is ‘boiling over with rage‘ or ‘steam is coming out of their ears’: understanding anger through containment of liquids. These are English language examples, but the principles seem to be universal. Cognitive scientists would argue that this perception of up and down is a very fundamental basic level category or building block of thought used to make sense of other, more abstract things. Embodied interaction seeks to make physical designs make sense to us through harnessing the way we understand the world through physical experience. At the moment, much tangible interaction work is happening in R&D labs...
Confidential Sunday, 10 April 2011 MIT’s information design ecology unit’s Proverbial Wallet aims to ‘unabstract virtual assets’ and provide a tangible link to virtual money. Connects to phone via Bluetooth, and then to your online banking service. It has an interface through physical actuators, but no screen. It gets fatter or thinner depending on how much money you have in the bank - direct physical representation of a metaphor (fat wallet). The hinge resistance increases when you have less money, making it harder to open. This is a nice example of encouraging positive behaviour without controlling the user. [metaphor of good angel?] The wallet buzzes when your bank processes a transaction, creating a conscious connection between handing over your credit card and the money actually leaving your account. This is a nice example of seamful design... communicating interconnections between services back to the user. http://eco.media.mit.edu/proverbialwallets/
| Confidential image: www.fabianhemmert.com Sunday, 10 April 2011 This is a prototype phone-shaped box by the tangible design researcher Fabian Hemmert at Deutsche Telekom labs. It has an actuated back plate (controlled by servo motors) which can be tilted. so it can be thin in your pocket, but tapered downwards when held in the hand. this can also be used to convey contextual information: for example to make the phone thick or thin on one side when reading an ebook to indicate how much has been read and is still to be read. the shape of the phone can also be used to indicate direction when providing navigation... http://fabianhemmert.com/projects/shape-changing-mobiles There’s a lot we can do with tangible design to convey information without screens. As digital designers who’ve only worked with screens, we should start to think about these new interaction channels and how they can help us reclaim digital interactions into the real world. Also, physical things are fun :)
wrong Sunday, 10 April 2011 Technology is not reliable... we were trying to share files with a ubicomp researcher over Bluetooth the other day and we couldn’t even get that working or figure out why it wasn’t working. Imagine that scaled up to a smarthome consisting of hundreds or thousands of connected objects... A friend’s Danish friend lives in a smarthome with a system to decides when you could or could not open windows. it broke. Thankfully, there was an override. Then the override failed, and the house took over, opening and closing windows on its own. No-one could figure out how to fix it. So the more interconnected things are, the harder it is to know which bit’s gone wrong. and the harder it is for the system to give useful information on overcoming errors.
2011 If technology becomes a part of your everyday life, reliability is critical. “A place where one can solve one’s worries” – a Korean euphemism for toilet In Korea, there are toilets with sensors for female hormones and the signs of diabetes. Right now, they don’t seem to include a pregnancy test but it’s not too much of a stretch to imagine. Home pregnancy tests are 99% accurate. Should you decide to test yourself once in that year, there’s only a 1 in 100 chance you’ll get a false positive. If you’re tested 1000 times a year by a smart toilet, your toilet will wrongly congratulate you almost once a month. You’d stop taking it seriously, wouldn’t you? How much trust would you place in these things? How dependent will we become along them?
April 2011 As the boundaries between the physical and digital world blur, we’ll see digital business models starting to appear in the physical world. Some of these may be more or less acceptable to users...
Sunday, 10 April 2011 People will pay more for a cheap toy if a digital service is added. Engagement models last longer - longer engagement with product and customer Ability to upsell over time, and have a closer direct relationship with the customer
open after this message from our sponsors! Sunday, 10 April 2011 Rethink how we own objects Freemium services - get basic service free with ads, pay for ad free. Will we see this extended to everyday objects? Will Tesco provide fridges free to anyone willing to share all their food consumption data, buy food from Tesco, and watch adverts on a regular basis?
a demographic image: kihonmedia/ compensationclaims.com Sunday, 10 April 2011 Pay for what you’re using/what you need, e.g. sensors in cars enable drivers to pay car insurance based on how they actually drive, not on how the average person of their demographic drives.
image: mp.natlib.govt.nz image: techpin.com Sunday, 10 April 2011 We recently interviewed some designers of smart objects as part of our project, about their methods. Numbers were small, but tendency was to feel that insight research was difficult to do before prototyping. We often can’t predict the changing behaviours and attitudes that will emerge from new technology until people actually have it. Putting washing machines in homes, plus bathrooms, changed ideas of acceptable hygiene - expectations of cleanliness rose. People didn’t used to get mobiles for social reasons, they claimed it was for emergencies, but once they had them they quickly adapt to using them for social reasons. (e.g. Edwards and Grinter). But observing current tasks can help us undersatnd cultural values, may not always help generate detailed requirements for highly novel systems, and you certainly can’t ask people what they want from something totally new. But there may be methods we can use...
April 2011 Participatory drama methods can be used to help shape designs... for example giving users scenarios to imagine novel interactions with an everyday object, invent interactions with an ambiguous object, or a wand that allows them to imbue other objects with magical powers. Also bodystorming - acting as if the product existed.
2011 Prototyping is an important part of exploration in tangible design. Need early stage iteration as it’s hard to change complex systems! but hard to create real enough experiences. Getting into hardware is a little daunting but there are ways to get feedback before going that far. You *can* use paper models to mock up and get feedback on quick ideas/key interactions, wizard of oz style. Also useful for concepting. You can also use video to mock up things that don’t yet exist to get feedback... this Microsoft Office 2019 one is for marketing but you can make versions in ways that support concept testing. Need to work closely with technologists... helps for designers to get some understanding of tech, and for developers to move closer to understanding user needs.
2011 Testing methods are geared towards one user, one task, one(ish) devices, that the user is highly focused on. We don’t even have a good definition of usability for more ambient interactions. With testing, ecological validity is an issue: you can do very controlled simulations in the lab which are good for key interactions, but when you’re designing something to work in an environment context of use is really important. Need to study usage over time as well - services are dynamic, and people may react differently to things once they become normal than they will to novel technology. So context, and time are important. One solution to this is the paratype: a modified diary method (Abowd, 05). This involves a simulated interaction with a certain technological artifact within a specific setting of real social action, and documenting the effects of this combination. - focuses on the social - samples experience using a simulation in real social contexts, e.g. personal audio loop - memory device - allowed them to study 3rd party reactions to the context. There’s more work to do to develop methods here, but academic ubicomp literature is trying to tackle this issue.
Everything we’ve talked about is happening now, somewhere. We think this is about to affect the work that many of us do, even if just in small ways. We’d like to suggest a few ways in which UX designers can start to think about this.
existing projects... Sunday, 10 April 2011 Look for potential in other projects by thinking of the ecosystem beyond your service and how this might be leveraged to extend or enrich your service When everything around you has the potential to be hardware and run software, the challenge lies in looking for the relationships between objects and services that provide real value to the user experience. A simple example is to start with mobile. Mobile phones are essentially sensor platforms - many of them have GPs that tells them where they are, motion sensors that tell them when they are moving, and some have compasses so they know which way they are facing. They also know what time it is, and can take pictures with the camera. If you’re delivering something on a mobile, even if it’s a website, you can usually access this data from the phone. How can you use that to make the service better?
April 2011 A possible starter for creating design briefs... Use objects as a basis for concepting... if they could tell a story, what would it be? what personality would they have? what emotions would they feel? what information would they know, or share? how would they talk to us? how could they have good manners? (i.e. some rules for interaction)
Be open to other skillsets. service design, interaction and product design will have to work together to address new challenges We’ll be doing a lot more work in this area over the next couple of years and would love to connect with anyone else who’s interested.
Ubiquitous Computing User Experience Design” Timo Arnall: “Designing for an internet of things” (http://www.slideshare.net/tmo/designing-for-an-internet-of-things) ReadWriteWeb: Internet of Things: http://www.readwriteweb.com/archives/internet-of-things/ Adam Greenfield: “Everyware: the dawning age of ubiquitous computing” Bruce Sterling: “Shaping Things” BERG London: http://berglondon.com/blog/ Some reading... Sunday, 10 April 2011
Alex von Feldmann, Dom Quigley, Ann Light, Alfred Lui, Ji-Hye Park, Sam Crosland, Martin Charlier, Helen Le Voi PS: we’re hiring in London Sunday, 10 April 2011
clurr
zakkerooni
otanet
godlingkogami
elsh
cyberagentdevelopers
usagimaru
ashapiro
zozodevelopers
kubosho
oguemon
cpineda57
sergeychernyshev
tammyeverts
phodgson
jeffersonlam
philnash
scottboms
akosma
wjessup
lynnandtonic
panda_program
mraible
![Thank you. @fjord [email protected] [email protected] / @clurr Thanks also to](https://files.speakerdeck.com/presentations/4fb7f7609720b900220026d2/slide_63.jpg){kind=link}