Devices, Browsers & You

A look at how we can interact with device hardware directly from a web browser. Presented at the first Swindon Geek Night.

Transcript

1. DEVICES, BROWSERS AND YOU
2. PETER NEWNHAM @wrakky blarg.co.uk
3. A BRIEF HISTORY
4. USER INTERACTION HTML controls < i n p u t > , < s e l e c t > , < t e x t a r e a > Javascript events Keyboard - o n k e y d o w n , o n k e y u p , e t c Mouse - o n c l i c k , o n m o u s e o v e r , e t c Touch - t o u c h s t a r t , t o u c h m o v e , e t c
5. BUT I WANT MORE! Install separate program Use a plugin Flash Java applet Silverlight
6. HOW DO WE GET AROUND THIS?
7. HTML5 New elements < v i d e o > , < a u d i o > , < c a n v a s > Native APIs Access device hardware directly from javascript Write once, run anywhere (ish) Still at the mercy of cross browser compatibility Chrome + Firefox around 65% statcounter.com
8. Navigator.getUserMedia Camera Microphone Screen sharing User permission required
9. BROWSER SUPPORT Desktop Mobile
10. EXAMPLE v a r g e t U s e r M e d i a = n a v i g a t o r . g e t U s e r M e d i a | | n a v i g a t o r . w e b k i t G e t U s e r M e d i a | | n a v i g a t o r . m o z G e t U s e r M e d i a ; g e t U s e r M e d i a ( { a u d i o : t r u e , v i d e o : t r u e / / c a n a l s o b e a c o n f i g o b j e c t } , f u n c t i o n ( s t r e a m ) { v a r v i d e o = d o c u m e n t . q u e r y S e l e c t o r ( ' v i d e o ' ) ; v i d e o . s r c = w i n d o w . U R L . c r e a t e O b j e c t U R L ( s t r e a m ) ; v i d e o . o n l o a d e d m e t a d a t a = f u n c t i o n ( e ) { v i d e o . p l a y ( ) ; } ; } , f u n c t i o n ( e r r ) { c o n s o l e . l o g ( " T h e f o l l o w i n g e r r o r o c c u r r e d : " + e r r . n a m e ) ; } ) ;
11. EXAMPLE Inline webcam example here
12. CAMERA
13. USES Video chat, conferencing Motion detection Facial recognition
14. FACIAL SUBSTITUTION DEMO http://auduno.github.io/clmtrackr/examples/facesubstitution.html clmtrackr WebGL
15. MICROPHONE
16. USES Phone calls Voice commands Speech to text Web Speech API - Chrome only Doppler effect http://danielrapp.github.io/doppler/
17. https://github.com/wrakky/dopply-bird
18. DEVICE ORIENTATION ACCELEROMETER, GYROSCOPE & MAGNOMETER
19. ORIENTATION alpha the number of degrees the device is being twisted around the center of the screen gamma the number of degrees the device is turned left or right beta the number of degrees the device is tipped forward or backward
20. EXAMPLE w i n d o w . a d d E v e n t L i s t e n e r ( " d e v i c e o r i e n t a t i o n " , f u n c t i o n ( e v e n t ) { v a r a l p h a = e v e n t . a l p h a ; / / d e g r e s s t w i s t e d a r o u n d c e n t e r v a r b e t a = e v e n t . b e t a ; / / d e g r e e s t i p p e d f o r w a r d o r b a c k w a r d v a r g a m m a = e v e n t . g a m m a ; / / d e g r e e s t u r n e d l e f t o r r i g h t / / d o s o m e t h i n g a w e s o m e h e r e } , t r u e ) ;
21. BROWSER SUPPORT Desktop Mobile
22. DEMO http://www.html5rocks.com/en/tutorials/device/orientation/de
23. GEOLOCATION Access user latitude and longitude Permission required
24. BROWSER SUPPORT Desktop 9+ Mobile
25. EXAMPLE n a v i g a t o r . g e o l o c a t i o n . g e t C u r r e n t P o s i t i o n ( s u c c e s s , e r r o r ) ; f u n c t i o n s u c c e s s ( p o s i t i o n ) { v a r l a t i t u d e = p o s i t i o n . c o o r d s . l a t i t u d e ; v a r l o n g i t u d e = p o s i t i o n . c o o r d s . l o n g i t u d e ; / / d o s o m e t h i n g c l e v e r h e r e } f u n c t i o n e r r o r ( ) { c o n s o l e . l o g ( ' U n a b l e t o g e t l o c a t i o n ' ) ; }
26. USES Find nearest [something] Plot user location on map - real time tracking Directions - to/from current location Geocaching
27. LIGHT SENSOR Gives light intensity value in lux 10 ~ 50 lux: Dim Environment 100 ~ 1000 lux: Normal > 10000 lux: Bright
28. USES Change page theme depending on light level
29. BROWSER SUPPORT Desktop Mobile
30. EXAMPLE w i n d o w . a d d E v e n t L i s t e n e r ( ' d e v i c e l i g h t ' , f u n c t i o n ( e v e n t ) { v a r l u m i n o s i t y = e v e n t . v a l u e ; / / d o s o m e t h i n g b l i n d i n g l y g o o d h e r e } ) ;
31. DEMO CURRENT LUX LEVEL
32. BEST OF THE REST
33. PROXIMITY Detect when a user is close to the screen Firefox only :( w i n d o w . a d d E v e n t L i s t e n e r ( ' u s e r p r o x i m i t y ' , f u n c t i o n ( e v e n t ) { i f ( e v e n t . n e a r ) { c o n s o l e . l o g ( ' u s e r i s n e a r ' ) ; } e l s e { c o n s o l e . l o g ( ' u s e r i s n o t n e a r ' ) ; } } , t r u e ) ;
34. VIBRATION Chrome, Firefox and Opera / / V i b r a t e o n c e f o r o n e s e c o n d n a v i g a t o r . v i b r a t e ( 1 0 0 0 ) ; / / V i b r a t e f o r 3 s e c o n d s , w a i t 2 s e c o n d s , t h e n v i b r a t e f o r 1 s e c o n d n a v i g a t o r . v i b r a t e ( [ 3 0 0 0 , 2 0 0 0 , 1 0 0 0 ] ) ;
35. BATTERY STATUS Chrome, Firefox and Opera n a v i g a t o r . g e t B a t t e r y ( ) . t h e n ( f u n c t i o n ( b a t t e r y ) { b a t t e r y . c h a r g i n g ; / / t r u e / f a l s e b a t t e r y . l e v e l ; / / % d i s p l a y e d a s d e c i m a l f r o m 0 t o 1 b a t t e r y . c h a r g i n g T i m e ; / / t i m e s p e n t c h a r g i n g b a t t e r y . d i s c h a r g i n g T i m e ; / / t i m e s p e n t n o t c h a r g i n g } ) ;
36. MORE COMING (ONE DAY) Temperature Humidity Air Pressure Noise Level
37. THE END