Mobile ♥ JavaScript - Hardware Access & Device APIs

My slides for JS Conf 2013.

Intro and some demo of HTML5's JavaScript APIs to access devices. APIs including: device orientation events, media capture, getUserMedia, battery status, vibration etc.

The interactive slide deck in HTML is available at:
https://dl.dropboxusercontent.com/u/1330446/presentation-slides/html5-mobile-approach/deviceAPIs.html

Transcript

1. Mobile S JavaScript Hardware Access & Device APIs Tomomi Imura May 29, 2013 @JS Conf
2. H E L L O M Y N A M E I S Tomomi @girlie_mac 02
6. URI Schemes: Making a Call < a h r e f = " t e l : + 1 4 1 5 5 5 5 7 7 7 7 " > O r d e r P i z z a N o w ! < / a > < a h r e f = " s m s : + 1 4 1 5 5 5 5 8 8 8 8 ? b o d y = O % 2 0 H a i " > T e x t m e ! < / a > 06
7. URI Schemes: UX 07
8. Geolocation Hardware has more than a few ways to detect your location: • GPS satellites • GSM/CDMA cell IDs • A-GPS • Wi-Fi base stations 08
9. Geolocation i f ( n a v i g a t o r . g e o l o c a t i o n ) { n a v i g a t o r . g e o l o c a t i o n . g e t C u r r e n t P o s i t i o n ( s u c c e s s , f a i l ) ; } f u n c t i o n s u c c e s s ( p o s i t i o n ) { a l e r t ( ' L a t i t u d e : ' + p o s i t i o n . c o o r d s . l a t i t u d e + ' , L o n g i t u d e : ' + p o s i t i o n . c o o r d s . l o n g i t u d e ) ; } 09
10. Access Permission 10
11. Device Orientation Events DOM events for obtaining information about the physical orientation and movement of the hosting device • Gyroscope • Accelerometer • Compass 11
12. Device Orientation Events Events: d e v i c e o r i e n t a t i o n, d e v i c e m o t i o n, c o m p a s s n e e d s c a l i b r a t i o n Dev Opera (Diagram) 12
13. Device Orientation Events Demo Watch the demo video on Vemeo: http://player.vimeo.com/video/51157652 Try it on supported browsers: http://goo.gl/5Cj4d 13
14. i f ( w i n d o w . D e v i c e O r i e n t a t i o n E v e n t ) { w i n d o w . a d d E v e n t L i s t e n e r ( ' d e v i c e o r i e n t a t i o n ' , f u n c t i o n ( e ) { a = M a t h . f l o o r ( e . a l p h a ) ; b = M a t h . f l o o r ( e . b e t a ) ; g = M a t h . f l o o r ( e . g a m m a ) ; e l . s t y l e . t r a n s f o r m = ' r o t a t e Z ( ' + a + ' d e g ) r o t a t e X ( ' + b + ' d e g ) r o t a t e Y ( ' + g + ' d e g ) ' ; . . . } , t r u e ) ; } 4.2 3.0 18 10 14 12 2.1 14
15. User Cases Can we re-create this iOS app in HTML5/JS? http://pizza-compass.com/ 15
16. Pizza Compass in JavaScript 1. Get location - navigator.geolocation 2. Use compass - deviceorientation event (use a l p h a) 3. Order a pizza - tel: URI scheme 16
17. Media Capture and Streams API "Web Real Time Communications" v a r g u m = n a v i g a t o r . g e t U s e r M e d i a | | n a v i g a t o r . m o z G e t U s e r M e d i a | | n a v i g a t o r . w e b k i t G e t U s e r M e d i a ; n a v i g a t o r . g e t U s e r M e d i a ( { v i d e o : t r u e , a u d i o : t r u e } , s u c c e s s C a l l b a c k , e r r o r C a l l b a c k ) ; 10 26* ** not supported by Opera 14 for Android (WebKit) 17* 2.1* 17
18. on Chrome 27 for Android The super silly demo: RespongeBob Web / Make Me Bob! 18
19. HTML Media Capture < ! - - C u r r e n t i m p l e m e n t a t i o n ( i f s u p p o r t e d ) - - > < i n p u t t y p e = " f i l e " a c c e p t = " i m a g e / * " c a p t u r e = " c a m e r a " > < ! - - N e w l y p r o p o s e d s p e c i f i c a t i o n - - > < i n p u t t y p e = " f i l e " a c c e p t = " i m a g e / * " c a p t u r e > 18 3 10 2.1 6** 10** ** Partial support: 'capture' attr not supported 10** 19
20. Watch the demo video: http://sdrv.ms/UF55gM Github: Coremob Camera: https://github.com/coremob/camera 1:18 20
21. Camera App in HTML5 1. Take a picture with the native camera via HTML Media Capture 2. The camera returns the pictures as a File object from FileReader() 3. drawImage() to draw the image object in canvas 4. getImageData() to obtain an ImageData object containing a copy of the pixel data for a context, then tweak the pixels (filter effect) 5. canvas.toBlob() to store the blob locally with IndexedDB* 6. Upload the final photo with XHR2/CORS 21
22. github.com/coremob/camera  22
23. Battery Status API v a r b a t t e r y = n a v i g a t o r . b a t t e r y | | n a v i g a t o r . w e b k i t B a t t e r y ; b a t t e r y . a d d E v e n t L i s t e n e r ( ' c h a r g i n g c h a n g e ' , u p d a t e S t a t u s ) ; b a t t e r y . a d d E v e n t L i s t e n e r ( ' l e v e l c h a n g e ' , u p d a t e S t a t u s ) ; f u n c t i o n u p d a t e S t a t u s ( ) { a l e r t ( ' B a t t e r y s t a t u s : ' + b a t t e r y . l e v e l * 1 0 0 + ' % ' ) ; i f ( b a t t e r y . c h a r g i n g ) { a l e r t ( ' B a t t e r y i s c h a r g i n g . . . ' ) ; } } 14* 16 2.1* 23
24. Battery Status API http://goo.gl/V1n6h (Pictured: Firefox) 24
25. Vibration API v a r v i b r a t e = n a v i g a t o r . v i b r a t e | | n a v i g a t o r . m o z V i b r a t e ; v i b r a t e ( 1 0 0 0 ) ; / / v i b r a t e f o r 1 s e c v i b r a t e ( [ 1 0 0 0 , 5 0 0 , 2 0 0 0 ] ) ; / / v i b r a t e s f o r 1 s e c , s t i l l f o r 0 . 5 s e c o n d s , / / a n d v i b r a t e s a g a i n f o r 2 s e c 14* 16 2.1 Demo on JSFiddle: http://goo.gl/EWPmL 25
26. Ambient Light Events w i n d o w . a d d E v e n t L i s t e n e r ( ' d e v i c e l i g h t ' , f u n c t i o n ( e ) { a l e r t ( e . v a l u e ) ; / / v a l u e i n d o u b l e } ) ; returned value (in lux) My Observation on Galaxy Nexus < 400 Indoor 400-1000 Office lighting. Outdoor (in foggy San Francisco) > 1000 Outdoor daylight (anywhere else in California) 15 26
27. Ambient Light Events w i n d o w . a d d E v e n t L i s t e n e r ( ' l i g h t l e v e l ' , f u n c t i o n ( e ) { a l e r t ( e . v a l u e ) ; / / v a l u e i n s t r i n g } ) ; returned value Illuminance Description dim < 50 lux dark enough that the light produced by a white background is eye-straining or distracting normal 50-10000 lux office building hallway, very dark overcast day, office lighting, sunrise or sunset on a clear day, overcast day, or similar bright > 10000 lux direct sunlight, or similarly bright conditions that make it hard to see things that aren't high-contrast 27
28. Light Sensor Demo Watch the demo video on Vemeo: http://player.vimeo.com/video/51322638 Try it on Firefox Mobile: http://goo.gl/WynKX 28
29. Proximity Events The distance of a nearby physical object using the proximity sensor of a device. Events: d e v i c e p r o x i m i t y and u s e r p r o x i m i t y w i n d o w . a d d E v e n t L i s t e n e r ( ' d e v i c e p r o x i m i t y ' , f u n c t i o n ( e ) { a l e r t ( e . v a l u e ) ; } ) ; 29
30. More... • The Network Information API • Clipboard API ( c u t, c o p y, p a s t e events) • Network Service Discovery (UPnP multi-media device control) • Fullscreen API • Gamepad API • Push API (Push Notification) 30
31. Uncertain APIs Bye-bye Web Intents • Contacts APIs • Calendar APIs • ... Hello, Web Activities? 31
32. Thank you! Tomomi Imura • girliemac.com • twitter.com/girlie_mac • github.com/girliemac 32
33. Notes People, who has not attended my talks but stumbled upon my slides sometimes complains (in passive-aggressive manner) that my browser stats are wrong - I have to tell you that all the browsers I am talking about are mobile! Even if you are so sure that certain features are supported on desktop, it may not be true for the mobile version. If you spot my mistakes on mobile browsers, just let me know :-) xoxo, 33