Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Building a Device Lab

245cee81a9c424266e5e401d844ea881?s=47 Lara Hogan
November 12, 2014

Building a Device Lab

by Lara Hogan and Destiny Montague
http://larahogan.me/devicelab/

245cee81a9c424266e5e401d844ea881?s=128

Lara Hogan

November 12, 2014
Tweet

Transcript

  1. Building a Device Lab Destiny Montague and Lara Hogan co-LAB-orators

    at Etsy Slides and links: larahogan.me/devicelab/
  2. Reminder: have your visitor’s  device data ready!

  3. What is Etsy? (and who are we?)

  4. Engineering Teams at Etsy • Feature Teams • Listing page

    • Seller tools • Checkout
  5. Engineering Teams at Etsy • Feature Teams • Listing page

    • Seller tools • Checkout • Infrastructure Teams • Performance • Dev tools • Security Destiny
  6. Why’d we build a device lab?

  7. traffic to Etsy from mobile web+apps: > 50%

  8. How many times have we built a device lab?

  9. v1: Gray Cabinet v2: Bookshelves v3: Leaning Tower Our device

    lab’s history
  10. Our device lab by the numbers • 36 devices •

    8 operating systems • 45 average users/month • 4 median checkouts/device • 8 devices retired this year, 
 12 new purchased v3: Leaning Tower
  11. a note about privilege

  12. what works for us  may not work for you

  13. What we’ll cover today 1. Choosing Devices (10 min) 2.

    Power (20 min) 3. Device Setup (10 min) 4. Setting Up Testing (5 min) 5. Networking, RFID Readers (15 min) 6. User Experience & Organization (15 min)
  14. Choosing Devices

  15. find your visitors’ devices

  16. None
  17. Device OS Version % Total Sessions Size Apple iPhone 7.1

    40% handset Apple iPad 7.1 7.9% small/large tablet Apple iPhone 7.0.6 4.9% handset LG Nexus 5 4.4.2 4.5% handset Google Nexus 4 4.4.2 4% handset Apple iPhone 7.0.4 3.2% handset Apple iPhone 7.1.1 3.2% handset Google Nexus 4 4.3.1 2.5% handset Google Nexus 7 4.4.2 2.1% small tablet Apple iPhone 6.1.3 1.5% handset
  18. Get the most bang for your buck

  19. focus on the 20% of devices that produce 
 80%

    of your traffic
  20. Device OS Version % Total Sessions Size Apple iPhone 7.1

    40% handset Apple iPad 7.1 7.9% small/large tablet Apple iPhone 7.0.6 4.9% handset LG Nexus 5 4.4.2 4.5% handset Google Nexus 4 4.4.2 4% handset Apple iPhone 7.0.4 3.2% handset Apple iPhone 7.1.1 3.2% handset Google Nexus 4 4.3.1 2.5% handset Google Nexus 7 4.4.2 2.1% small tablet Apple iPhone 6.1.3 1.5% handset
  21. cover your visitors’ screen sizes and operating systems

  22. Device OS Version % Total Sessions Size Apple iPhone 7.1

    40% handset Apple iPad 7.1 7.9% small/large tablet Apple iPhone 7.0.6 4.9% handset LG Nexus 5 4.4.2 4.5% handset Google Nexus 4 4.4.2 4% handset Apple iPhone 7.0.4 3.2% handset Apple iPhone 7.1.1 3.2% handset Google Nexus 4 4.3.1 2.5% handset Google Nexus 7 4.4.2 2.1% small tablet Apple iPhone 6.1.3 1.5% handset
  23. buggiest: Kindle, older Android

  24. Device OS Version % Total Sessions Size Apple iPhone 7.1

    40% handset Apple iPad 7.1 7.9% small/large tablet Apple iPhone 7.0.6 4.9% handset LG Nexus 5 4.4.2 4.5% handset Google Nexus 4 4.4.2 4% handset Apple iPhone 7.0.4 3.2% handset Apple iPhone 7.1.1 3.2% handset Google Nexus 4 4.3.1 2.5% handset Google Nexus 7 4.4.2 2.1% small tablet Apple iPhone 6.1.3 1.5% handset
  25. Does usage differ if you segment by audience region 

    or target market?
  26. combination of  touch and cursor

  27. retina/DPI

  28. Device OS Version % Total Sessions Size Apple iPhone 7.1

    40% handset Apple iPad 7.1 7.9% small/large tablet Apple iPhone 7.0.6 4.9% handset LG Nexus 5 4.4.2 4.5% handset Google Nexus 4 4.4.2 4% handset Apple iPhone 7.0.4 3.2% handset Apple iPhone 7.1.1 3.2% handset Google Nexus 4 4.3.1 2.5% handset Google Nexus 7 4.4.2 2.1% small tablet Apple iPhone 6.1.3 1.5% handset
  29. Device OS Version % Total Sessions Size Apple iPhone 7.1

    40% handset Apple iPad 7.1 7.9% small/large tablet Apple iPhone 7.0.6 4.9% handset LG Nexus 5 4.4.2 4.5% handset Google Nexus 4 4.4.2 4% handset Apple iPhone 7.0.4 3.2% handset Apple iPhone 7.1.1 3.2% handset Google Nexus 4 4.3.1 2.5% handset Google Nexus 7 4.4.2 2.1% small tablet Apple iPhone 6.1.3 1.5% handset
  30. purchasing devices

  31. used devices better- represent battery life

  32. wifi-capable vs. data plan

  33. eBay, Amazon,  local cell phone dealers

  34. our devices

  35. In our lab: • 12 Android phones (a range of

    operating system versions and major manufacturers) • 3 Android tablets (a range of screen resolutions) • 1 Blackberry phone • 1 Firefox phone (developer preview) • 5 iPads (including 1 mini, different generations)
  36. In our lab (cont’): • 5 iPhones/iPods • 1 Kindle

    Fires (range of screen resolutions) • 1 Windows Phone, 1 Windows Surface, 1 Dell laptop • 1 Google Glass, 1 Chromebook Pixel • 1 Raspberry Pi, and some Kubis
  37. More reading • Strategies for choosing test devices: 
 http://stephanierieger.com/strategies-for-choosing-test-

    devices/ • How to build a device lab, with a thorough first section on choosing and acquiring devices:
 http://dmolsen.com/2012/06/26/how-to-build-a- device-lab-part-1/
  38. questions?

  39. Power

  40. None
  41. electricity 101

  42. https://www.flickr.com/photos/marktee/9223003535 electricity = water

  43. https://www.flickr.com/photos/marktee/9223003535 usb/power cables = garden hose

  44. amps vs. volts

  45. amperage

  46. voltage

  47. volts * amps = watts!

  48. None
  49. power considerations

  50. how much power does each device draw, 
 according to

    the manufacturer's specs?
  51. how much power can the 
 surge protector  support?

  52. how much power can each usb hub support?

  53. None
  54. build to support  your most  power-hungry devices

  55. power problems

  56. power problems won’t accept a charge cycle on/off forever won’t

    turn on blinking battery can’t hold a charge someone stole the battery
  57. http://www.microsoft.com/surface/en-us/support/warranty-service-and-recovery/

  58. troubleshooting

  59. start simple

  60. "When you see hoof prints,  look for horses, 

    not zebras"
  61. isolate the problem

  62. DEVICE HARDWARE SOFTWARE

  63. DEVICE HARDWARE cables charging port peripherals SOFTWARE battery display

  64. DEVICE HARDWARE SOFTWARE apps operating system your base image cables

    charging port peripherals battery display
  65. why?

  66. •devices drew too much power ! •devices weren’t getting enough

    power ! •bad USB cables / peripherals ! •bad software ! •bad device
  67. iterating on power

  68. v1 - power strips

  69. v2 - stupid USB hubs

  70. v3 - better USB hubs

  71. Cambrionix hub

  72. Cambrionix hub • 32 ports • smart power delegation •

    fault indicator • disables problem devices
  73. v4 - better cabling/monitoring

  74. energy consumption

  75. None
  76. modlet timer

  77. None
  78. pro power tips

  79. http://iheartorganizing.blogspot.com/2012/03/you-asked-organizing-cords.html

  80. have spares/duplicates as back-ups

  81. have extra cables handy for people to steal

  82. questions?

  83. Device Setup

  84. configuration

  85. factory (re)format

  86. disable  auto-updates

  87. create an  app store account

  88. install apps

  89. restrict devices from syncing to computers

  90. set browser default homepage

  91. make use of wallpaper

  92. set up a passcode

  93. have a way to identify devices

  94. have a backup image

  95. enroll devices in a  mobile device management console (MDM)

  96. choosing your MDM

  97. scalability

  98. app management

  99. admin console

  100. security

  101. http://www.enterpriseios.com/wiki/Comparison_MDM_Providers MDM providers

  102. questions?

  103. Testing

  104. None
  105. None
  106. 0. set up an Adobe account with the device lab

     email address ! (save this password somewhere;  you’re gonna need it later.)
  107. 1. install the Adobe Edge Inspect app via Creative Cloud

    desktop app ! (yeah, you need them both. and it has to be running for everything else to work.)
  108. 2. install the  browser  extension ! (it’ll remind

    you to turn on the  application if you haven’t yet.)
  109. 3. install the app on  iOS, Android and 

    Kindle devices ! (right, it’s not on every operating system yet, and this is a great case for getting an MDM.)
  110. 4. enter the pin for  each device in the

    browser extension ! (you’ll have to upgrade to test on  multiple devices at once.) http://www.adobe.com/inspire/2012/12/web-designs-edge-inspect.html
  111. if you upgrade your account, you’ll want to reuse it

    on every laptop used for testing...
  112. which is why it’s great to keep one laptop in

    the lab for this express purpose.
  113. None
  114. save all the  screenshots at once! http://www.adobe.com/inspire/2012/12/web-designs-edge-inspect.html

  115. development machines & staging environments

  116. ensure it’s easy to  test development on your devices

  117. ensure it’s easy to  test all environments on 

    your devices
  118. None
  119. questions?

  120. Networking

  121. get connected

  122. use wifi

  123. bypass activation

  124. http://www.droidxforums.com/forum/droid-x-faqs/506-bypass-android-activation-screen-motorola-droid-x.html B A

  125. designate your  own network

  126. private SSID

  127. Guest Network Internal Network Device Lab Network Access - basic

    web - no internal sites - VPN is blocked ! - basic web - access to internal sites - some sites require VPN access to prod network - basic web - only pre-production/ testing & VM stuff that is essential for testing our internal wifi networks
  128. networking problems

  129. decrease in  wireless range

  130. decrease in data throughput

  131. intermittent or dropped wifi connection

  132. difficulty pairing  with bluetooth

  133. signal-to-noise ratio

  134. signal-to-noise ratio

  135. None
  136. RFID <3

  137. None
  138. what is RFID?

  139. “Wireless non-contact use of radio-frequency electromagnetic fields to transfer data.”

  140. None
  141. high or low frequency

  142. employee badges

  143. usefulness

  144. links to employee

  145. saves time

  146. can provide stats and trends

  147. less expensive

  148. components

  149. office h4x0rz

  150. RFID tags

  151. RFID reader

  152. raspberry pi

  153. LCD screen, keyboard, misc cables

  154. the stump by Trish Czech of “Timid Monsters”

  155. None
  156. None
  157. None
  158. problems

  159. interference

  160. identifying input

  161. questions?

  162. User Experience

  163. surprises

  164. surprises changed passcode left unplugged broken screen stolen cables lost

    devices connected personal facebook account changed wallpaper changed network connected personal email address upgraded operating system stolen stands stolen battery
  165. make it easy to  plug devices back in

  166. cable drops, hooks or ties

  167. mark cables and hubs

  168. make stealing cables hard

  169. Access to power In v2, power strips were very accessible

    so that we could watch the timed power strip for greenness.
  170. Access to power In v3 we hid the power strips

    so users can’t easily steal cables and the lab looks more organized. We still made it easy for our team to go in and 
 re-cable devices.
  171. make it easy to  stand devices up for testing

    & returning
  172. purchase a range of sizes

  173. not one-size-fits-all

  174. or, build-in stands

  175. give people  other things to steal

  176. Group your devices There are many ways to organize your

    devices. Be consistent, such as grouping by: • device size (handsets, small tablets, large tablets, etc.) • operating system • unique devices on the top shelf (Google Glass, Firefox OS, Windows phone)
  177. Labels on devices and shelves Make it easy to match

    devices to their home base. • Label the front of the shelf and the back of the device with the same wording. • Color-code by operating system so it’s easy to scan.
  178. Device information For each device, label nearby: • device name

    and ID number • screen resolution, pixel density • operating system version
  179. None
  180. Keep major passwords a secret The device unlock passcodes should

    all be the same and well-known. But don’t tell anybody: • app store passwords • email passwords • WiFi network password (if you’re using a subnet specifically for the device lab)
  181. check the lab regularly

  182. plugged in & charging

  183. missing devices

  184. unlock to find surprises

  185. usability tips

  186. Document a how-to • Passcodes (the same for every device)

    • Instructions for using Adobe Edge Inspect • Instructions for testing on VMs/staging • Instructions for taking screenshots and checking email on various operating systems
  187. Email on all devices • Set up a common email

    address (devicelab@) for testing email sends • Install a default email client and any other major email apps for testing • Configure it to receive all devicelab@ emails
  188. Put reminders everywhere You’ll need to constantly remind people to

    not upgrade apps or OSes, and to check out devices. • as background images across devices • on the wall next to the lab • on all wiki docs
  189. Make it easy to check out devices • Library card

    underneath 
 each device (color-coded, 
 of course) • RFID tags on each device and reader that works with employee badges
  190. Make it easy to test • Set up a shared

    laptop to connect with Adobe Edge Inspect • you only have to enter all the pins once • use a laptop (like Windows!) that can stay in your lab • Make sure VMs and staging environments can be accessed
  191. Make it easy to collaborate • Set up comfortable seating

    nearby • Nearby long tables can be handy, too • Make sure other outlets are available for people to plug in their own laptops • Make it so that people don’t have to walk back to their desks to test their work
  192. Make it attractive A good-looking lab is inviting. • New

    hires see it on their tour and guests want to check it out • No one wants to be near clutter • The more organized, the more likely they’ll put stuff back properly and want to use it
  193. examples

  194. Physical shelving: v2 1. Unique devices 2. iOS devices 3.

    Kindles and Android tablets 4. Android handsets 5. Crates and power
  195. Physical shelving: v2 • Cable drops for organization • Library

    cards and washi tape for labels • Different-height shelves
  196. Physical shelving: v3 1. Unique devices 2. Kindles | iOS

    tablets 3. Android | iOS handsets 4. Android | iOS handsets 5. Crates and misc.
  197. Physical shelving: v3 • Cables & power hidden behind shelving

    • Internal dashboard for device info and checkouts • Different-height shelves
  198. Thank you! larahogan.me/devicelab/ @lara_hogan @thisiscarlsagan