Building a Device Lab - Speaker Deck

Slide 1

Slide 1 text

Building a Device Lab Destiny Montague and Lara Hogan co-LAB-orators at Etsy Slides and links: larahogan.me/devicelab/

Slide 2

Slide 2 text

Reminder: have your visitor’s device data ready!

Slide 3

Slide 3 text

What is Etsy? (and who are we?)

Slide 4

Slide 4 text

Engineering Teams at Etsy • Feature Teams • Listing page • Seller tools • Checkout

Slide 5

Slide 5 text

Engineering Teams at Etsy • Feature Teams • Listing page • Seller tools • Checkout • Infrastructure Teams • Performance • Dev tools • Security Destiny

Slide 6

Slide 6 text

Why’d we build a device lab?

Slide 7

Slide 7 text

traffic to Etsy from mobile web+apps: > 50%

Slide 8

Slide 8 text

How many times have we built a device lab?

Slide 9

Slide 9 text

v1: Gray Cabinet v2: Bookshelves v3: Leaning Tower Our device lab’s history

Slide 10

Slide 10 text

Our device lab by the numbers • 36 devices • 8 operating systems • 45 average users/month • 4 median checkouts/device • 8 devices retired this year,   12 new purchased v3: Leaning Tower

Slide 11

Slide 11 text

a note about privilege

Slide 12

Slide 12 text

what works for us may not work for you

Slide 13

Slide 13 text

What we’ll cover today 1. Choosing Devices (10 min) 2. Power (20 min) 3. Device Setup (10 min) 4. Setting Up Testing (5 min) 5. Networking, RFID Readers (15 min) 6. User Experience & Organization (15 min)

Slide 14

Slide 14 text

Choosing Devices

Slide 15

Slide 15 text

find your visitors’ devices

Slide 16

Slide 16 text

No content

Slide 17

Slide 17 text

Device OS Version % Total Sessions Size Apple iPhone 7.1 40% handset Apple iPad 7.1 7.9% small/large tablet Apple iPhone 7.0.6 4.9% handset LG Nexus 5 4.4.2 4.5% handset Google Nexus 4 4.4.2 4% handset Apple iPhone 7.0.4 3.2% handset Apple iPhone 7.1.1 3.2% handset Google Nexus 4 4.3.1 2.5% handset Google Nexus 7 4.4.2 2.1% small tablet Apple iPhone 6.1.3 1.5% handset

Slide 18

Slide 18 text

Get the most bang for your buck

Slide 19

Slide 19 text

focus on the 20% of devices that produce   80% of your traffic

Slide 20

Slide 20 text

Slide 21

Slide 21 text

cover your visitors’ screen sizes and operating systems

Slide 22

Slide 22 text

Slide 23

Slide 23 text

buggiest: Kindle, older Android

Slide 24

Slide 24 text

Slide 25

Slide 25 text

Does usage differ if you segment by audience region or target market?

Slide 26

Slide 26 text

combination of touch and cursor

Slide 27

Slide 27 text

retina/DPI

Slide 28

Slide 28 text

Slide 29

Slide 29 text

Slide 30

Slide 30 text

purchasing devices

Slide 31

Slide 31 text

used devices better- represent battery life

Slide 32

Slide 32 text

wifi-capable vs. data plan

Slide 33

Slide 33 text

eBay, Amazon, local cell phone dealers

Slide 34

Slide 34 text

our devices

Slide 35

Slide 35 text

In our lab: • 12 Android phones (a range of operating system versions and major manufacturers) • 3 Android tablets (a range of screen resolutions) • 1 Blackberry phone • 1 Firefox phone (developer preview) • 5 iPads (including 1 mini, different generations)

Slide 36

Slide 36 text

In our lab (cont’): • 5 iPhones/iPods • 1 Kindle Fires (range of screen resolutions) • 1 Windows Phone, 1 Windows Surface, 1 Dell laptop • 1 Google Glass, 1 Chromebook Pixel • 1 Raspberry Pi, and some Kubis

Slide 37

Slide 37 text

More reading • Strategies for choosing test devices:   http://stephanierieger.com/strategies-for-choosing-test- devices/ • How to build a device lab, with a thorough first section on choosing and acquiring devices:  http://dmolsen.com/2012/06/26/how-to-build-a- device-lab-part-1/

Slide 38

Slide 38 text

questions?

Slide 39

Slide 39 text

Power

Slide 40

Slide 40 text

No content

Slide 41

Slide 41 text

electricity 101

Slide 42

Slide 42 text

https://www.flickr.com/photos/marktee/9223003535 electricity = water

Slide 43

Slide 43 text

https://www.flickr.com/photos/marktee/9223003535 usb/power cables = garden hose

Slide 44

Slide 44 text

amps vs. volts

Slide 45

Slide 45 text

amperage

Slide 46

Slide 46 text

voltage

Slide 47

Slide 47 text

volts * amps = watts!

Slide 48

Slide 48 text

No content

Slide 49

Slide 49 text

power considerations

Slide 50

Slide 50 text

how much power does each device draw,   according to the manufacturer's specs?

Slide 51

Slide 51 text

how much power can the   surge protector support?

Slide 52

Slide 52 text

how much power can each usb hub support?

Slide 53

Slide 53 text

No content

Slide 54

Slide 54 text

build to support your most power-hungry devices

Slide 55

Slide 55 text

power problems

Slide 56

Slide 56 text

power problems won’t accept a charge cycle on/off forever won’t turn on blinking battery can’t hold a charge someone stole the battery

Slide 57

Slide 57 text

http://www.microsoft.com/surface/en-us/support/warranty-service-and-recovery/

Slide 58

Slide 58 text

troubleshooting

Slide 59

Slide 59 text

start simple

Slide 60

Slide 60 text

"When you see hoof prints, look for horses, not zebras"

Slide 61

Slide 61 text

isolate the problem

Slide 62

Slide 62 text

DEVICE HARDWARE SOFTWARE

Slide 63

Slide 63 text

DEVICE HARDWARE cables charging port peripherals SOFTWARE battery display

Slide 64

Slide 64 text

DEVICE HARDWARE SOFTWARE apps operating system your base image cables charging port peripherals battery display

Slide 65

Slide 65 text

why?

Slide 66

Slide 66 text

•devices drew too much power ! •devices weren’t getting enough power ! •bad USB cables / peripherals ! •bad software ! •bad device

Slide 67

Slide 67 text

iterating on power

Slide 68

Slide 68 text

v1 - power strips

Slide 69

Slide 69 text

v2 - stupid USB hubs

Slide 70

Slide 70 text

v3 - better USB hubs

Slide 71

Slide 71 text

Cambrionix hub

Slide 72

Slide 72 text

Cambrionix hub • 32 ports • smart power delegation • fault indicator • disables problem devices

Slide 73

Slide 73 text

v4 - better cabling/monitoring

Slide 74

Slide 74 text

energy consumption

Slide 75

Slide 75 text

No content

Slide 76

Slide 76 text

modlet timer

Slide 77

Slide 77 text

No content

Slide 78

Slide 78 text

pro power tips

Slide 79

Slide 79 text

http://iheartorganizing.blogspot.com/2012/03/you-asked-organizing-cords.html

Slide 80

Slide 80 text

have spares/duplicates as back-ups

Slide 81

Slide 81 text

have extra cables handy for people to steal

Slide 82

Slide 82 text

questions?

Slide 83

Slide 83 text

Device Setup

Slide 84

Slide 84 text

configuration

Slide 85

Slide 85 text

factory (re)format

Slide 86

Slide 86 text

disable auto-updates

Slide 87

Slide 87 text

create an app store account

Slide 88

Slide 88 text

install apps

Slide 89

Slide 89 text

restrict devices from syncing to computers

Slide 90

Slide 90 text

set browser default homepage

Slide 91

Slide 91 text

make use of wallpaper

Slide 92

Slide 92 text

set up a passcode

Slide 93

Slide 93 text

have a way to identify devices

Slide 94

Slide 94 text

have a backup image

Slide 95

Slide 95 text

enroll devices in a mobile device management console (MDM)

Slide 96

Slide 96 text

choosing your MDM

Slide 97

Slide 97 text

scalability

Slide 98

Slide 98 text

app management

Slide 99

Slide 99 text

admin console

Slide 100

Slide 100 text

security

Slide 101

Slide 101 text

http://www.enterpriseios.com/wiki/Comparison_MDM_Providers MDM providers

Slide 102

Slide 102 text

questions?

Slide 103

Slide 103 text

Testing

Slide 104

Slide 104 text

No content

Slide 105

Slide 105 text

No content

Slide 106

Slide 106 text

0. set up an Adobe account with the device lab email address ! (save this password somewhere; you’re gonna need it later.)

Slide 107

Slide 107 text

1. install the Adobe Edge Inspect app via Creative Cloud desktop app ! (yeah, you need them both. and it has to be running for everything else to work.)

Slide 108

Slide 108 text

2. install the browser extension ! (it’ll remind you to turn on the application if you haven’t yet.)

Slide 109

Slide 109 text

3. install the app on iOS, Android and Kindle devices ! (right, it’s not on every operating system yet, and this is a great case for getting an MDM.)

Slide 110

Slide 110 text

4. enter the pin for each device in the browser extension ! (you’ll have to upgrade to test on multiple devices at once.) http://www.adobe.com/inspire/2012/12/web-designs-edge-inspect.html

Slide 111

Slide 111 text

if you upgrade your account, you’ll want to reuse it on every laptop used for testing...

Slide 112

Slide 112 text

which is why it’s great to keep one laptop in the lab for this express purpose.

Slide 113

Slide 113 text

No content

Slide 114

Slide 114 text

save all the screenshots at once! http://www.adobe.com/inspire/2012/12/web-designs-edge-inspect.html

Slide 115

Slide 115 text

development machines & staging environments

Slide 116

Slide 116 text

ensure it’s easy to test development on your devices

Slide 117

Slide 117 text

ensure it’s easy to test all environments on your devices

Slide 118

Slide 118 text

No content

Slide 119

Slide 119 text

questions?

Slide 120

Slide 120 text

Networking

Slide 121

Slide 121 text

get connected

Slide 122

Slide 122 text

use wifi

Slide 123

Slide 123 text

bypass activation

Slide 124

Slide 124 text

http://www.droidxforums.com/forum/droid-x-faqs/506-bypass-android-activation-screen-motorola-droid-x.html B A

Slide 125

Slide 125 text

designate your own network

Slide 126

Slide 126 text

private SSID

Slide 127

Slide 127 text

Guest Network Internal Network Device Lab Network Access - basic web - no internal sites - VPN is blocked ! - basic web - access to internal sites - some sites require VPN access to prod network - basic web - only pre-production/ testing & VM stuff that is essential for testing our internal wifi networks

Slide 128

Slide 128 text

networking problems

Slide 129

Slide 129 text

decrease in wireless range

Slide 130

Slide 130 text

decrease in data throughput

Slide 131

Slide 131 text

intermittent or dropped wifi connection

Slide 132

Slide 132 text

difficulty pairing with bluetooth

Slide 133

Slide 133 text

signal-to-noise ratio

Slide 134

Slide 134 text

signal-to-noise ratio

Slide 135

Slide 135 text

No content

Slide 136

Slide 136 text

RFID <3

Slide 137

Slide 137 text

No content

Slide 138

Slide 138 text

what is RFID?

Slide 139

Slide 139 text

“Wireless non-contact use of radio-frequency electromagnetic fields to transfer data.”

Slide 140

Slide 140 text

No content

Slide 141

Slide 141 text

high or low frequency

Slide 142

Slide 142 text

employee badges

Slide 143

Slide 143 text

usefulness

Slide 144

Slide 144 text

links to employee

Slide 145

Slide 145 text

saves time

Slide 146

Slide 146 text

can provide stats and trends

Slide 147

Slide 147 text

less expensive

Slide 148

Slide 148 text

components

Slide 149

Slide 149 text

office h4x0rz

Slide 150

Slide 150 text

RFID tags

Slide 151

Slide 151 text

RFID reader

Slide 152

Slide 152 text

raspberry pi

Slide 153

Slide 153 text

LCD screen, keyboard, misc cables

Slide 154

Slide 154 text

the stump by Trish Czech of “Timid Monsters”

Slide 155

Slide 155 text

No content

Slide 156

Slide 156 text

No content

Slide 157

Slide 157 text

No content

Slide 158

Slide 158 text

problems

Slide 159

Slide 159 text

interference

Slide 160

Slide 160 text

identifying input

Slide 161

Slide 161 text

questions?

Slide 162

Slide 162 text

User Experience

Slide 163

Slide 163 text

surprises

Slide 164

Slide 164 text

surprises changed passcode left unplugged broken screen stolen cables lost devices connected personal facebook account changed wallpaper changed network connected personal email address upgraded operating system stolen stands stolen battery

Slide 165

Slide 165 text

make it easy to plug devices back in

Slide 166

Slide 166 text

cable drops, hooks or ties

Slide 167

Slide 167 text

mark cables and hubs

Slide 168

Slide 168 text

make stealing cables hard

Slide 169

Slide 169 text

Access to power In v2, power strips were very accessible so that we could watch the timed power strip for greenness.

Slide 170

Slide 170 text

Access to power In v3 we hid the power strips so users can’t easily steal cables and the lab looks more organized. We still made it easy for our team to go in and   re-cable devices.

Slide 171

Slide 171 text

make it easy to stand devices up for testing & returning

Slide 172

Slide 172 text

purchase a range of sizes

Slide 173

Slide 173 text

not one-size-fits-all

Slide 174

Slide 174 text

or, build-in stands

Slide 175

Slide 175 text

give people other things to steal

Slide 176

Slide 176 text

Group your devices There are many ways to organize your devices. Be consistent, such as grouping by: • device size (handsets, small tablets, large tablets, etc.) • operating system • unique devices on the top shelf (Google Glass, Firefox OS, Windows phone)

Slide 177

Slide 177 text

Labels on devices and shelves Make it easy to match devices to their home base. • Label the front of the shelf and the back of the device with the same wording. • Color-code by operating system so it’s easy to scan.

Slide 178

Slide 178 text

Device information For each device, label nearby: • device name and ID number • screen resolution, pixel density • operating system version

Slide 179

Slide 179 text

No content

Slide 180

Slide 180 text

Keep major passwords a secret The device unlock passcodes should all be the same and well-known. But don’t tell anybody: • app store passwords • email passwords • WiFi network password (if you’re using a subnet specifically for the device lab)

Slide 181

Slide 181 text

check the lab regularly

Slide 182

Slide 182 text

plugged in & charging

Slide 183

Slide 183 text

missing devices

Slide 184

Slide 184 text

unlock to find surprises

Slide 185

Slide 185 text

usability tips

Slide 186

Slide 186 text

Document a how-to • Passcodes (the same for every device) • Instructions for using Adobe Edge Inspect • Instructions for testing on VMs/staging • Instructions for taking screenshots and checking email on various operating systems

Slide 187

Slide 187 text

Email on all devices • Set up a common email address (devicelab@) for testing email sends • Install a default email client and any other major email apps for testing • Configure it to receive all devicelab@ emails

Slide 188

Slide 188 text

Put reminders everywhere You’ll need to constantly remind people to not upgrade apps or OSes, and to check out devices. • as background images across devices • on the wall next to the lab • on all wiki docs

Slide 189

Slide 189 text

Make it easy to check out devices • Library card underneath   each device (color-coded,   of course) • RFID tags on each device and reader that works with employee badges

Slide 190

Slide 190 text

Make it easy to test • Set up a shared laptop to connect with Adobe Edge Inspect • you only have to enter all the pins once • use a laptop (like Windows!) that can stay in your lab • Make sure VMs and staging environments can be accessed

Slide 191

Slide 191 text

Make it easy to collaborate • Set up comfortable seating nearby • Nearby long tables can be handy, too • Make sure other outlets are available for people to plug in their own laptops • Make it so that people don’t have to walk back to their desks to test their work

Slide 192

Slide 192 text

Make it attractive A good-looking lab is inviting. • New hires see it on their tour and guests want to check it out • No one wants to be near clutter • The more organized, the more likely they’ll put stuff back properly and want to use it

Slide 193

Slide 193 text

examples

Slide 194

Slide 194 text

Physical shelving: v2 1. Unique devices 2. iOS devices 3. Kindles and Android tablets 4. Android handsets 5. Crates and power

Slide 195

Slide 195 text

Physical shelving: v2 • Cable drops for organization • Library cards and washi tape for labels • Different-height shelves

Slide 196

Slide 196 text

Physical shelving: v3 1. Unique devices 2. Kindles | iOS tablets 3. Android | iOS handsets 4. Android | iOS handsets 5. Crates and misc.

Slide 197

Slide 197 text

Physical shelving: v3 • Cables & power hidden behind shelving • Internal dashboard for device info and checkouts • Different-height shelves

Slide 198

Slide 198 text

Thank you! larahogan.me/devicelab/ @lara_hogan @thisiscarlsagan