Mobile HTML5 Use-Cases (CoreMob Camera App Showcase)

Showcasing the capabilities of the Web platform and more specifically of the subset of features that mobile platforms can take advantage of, by using an open source mobile app called Coremob Camera, using HTML5 to explain the real-life use cases of HTML5 in mobile. The technology behind the app is purely in HTML, CSS and JavaScript.

The HTML5 APIs covered in this talk includes HTML Media Capture, FileReader API, Canvas, IndexedDB, and XHR2/CORS.

The HTML version of this slide deck (original) is at: http://girliemac.github.io/presentation-slides/html5-mobile-usecases/

Transcript

1. Real Life HTML5 Use Cases for Mobile CoreMob Camera App
2. H E L L O M Y N A M E I S Tomomi @girlie_mac 02
4. Mobile (vs. Sessile) Luc Viatour | www.Lucnix.be • & • caitbiology.wikispaces.com
5. Communicating AP PHOTO / Kevin Lamarque
6. Reading http://www.flickr.com/photos/mi-ki/2454554920/
7. Playing http://www.flickr.com/photos/potyike/4599545293/
8. Taking Pictures http://www.flickr.com/photos/insane_capture/8113802393/
9. Locating http://www.flickr.com/photos/hezicohen/4648981682/
10. Paying http://www.flickr.com/photos/u-suke/100870738/
11. More... • Listening music, watching movies • Searching & browsing web • Chat, video calls • Note taking (Text, voice, camera input) • Scanning, OCR, Translations (AR) • etc. iphonelife.com 11
12. http://lemarquis.deviantart.com/art/HTML5-Wallpaper-243682854
13. Core Mobile Web Platform CG Goal: To accelerate the adoption of the Mobile Web as a compelling platform for the development of modern mobile Web applications http://coremob.github.com/coremob-2012/ 13
14. Core Mobile Web Platform CG 1. Agree on core features developers can depend on 2. Compile related conformance test suites* 3. Provide use cases, scenarios, and other input to drive successful mobile deployment http://coremob.github.com/coremob-2012/ 14
15. Coremob Camera
16. Watch the Video: http://sdrv.ms/UF55gM 1:18 16
17. Project Goals 1. Showcase the capabilities of the Web platform 2. Educate Web developers 3. Help improve browsers 17
18. HTML Media Capture Taking a photo with using a native camera < i n p u t t y p e = " f i l e " a c c e p t = " i m a g e / * " c a p t u r e = " c a m e r a " > / / N e w s p e c < i n p u t t y p e = " f i l e " a c c e p t = " i m a g e / * " c a p t u r e > 18 15bl 3 10 6* 10* 10* 18
19. File API Returning the photo as a f i l e object v a r i n p u t = d o c u m e n t . q u e r y S e l e c t o r ( ' i n p u t [ t y p e = f i l e ] ' ) ; c a m e r a . a d d E v e n t L i s t e n e r ( ' c h a n g e ' , f u n c t i o n ( ) { v a r l o c a l F i l e = i n p u t . f i l e s [ 0 ] ; v a r r e a d e r = n e w F i l e R e a d e r ( ) ; r e a d e r . r e a d A s D a t a U R L ( l o c a l F i l e ) ; r e a d e r . o n l o a d = f u n c t i o n ( e ) { p r e v i e w . s r c = e . t a r g e t . r e s u l t ; } } , f a l s e ) ; 19
20. Canvas Applying filters to the photo v a r c = d o c u m e n t . c r e a t e E l e m e n t ( ' c a n v a s ' ) ; v a r c t x = t h i s . c . g e t C o n t e x t ( ' 2 d ' ) ; c t x . d r a w I m a g e ( i m g O b j , 0 , 0 ) ; v a r i m g D a t a = c t x . g e t I m a g e D a t a ( x , y , w , h ) ; / / . . . P i x e l m a n i p u l a t i o n . . . c t x . p u t I m a g e D a t a ( i m g D a t a , 0 , 0 ) ; 7 9 20
21. Canvas Pixel Manipulation 21
22. Canvas Pixel Manipulation / / G r a y s c a l e u s i n g C I E 1 9 3 1 l u m i n a n c e v a r d = i m g D a t a . d a t a ; f o r ( v a r i = 0 ; i < d . l e n g t h ; i + = 4 ) { v a r r = d [ i ] ; v a r g = d [ i + 1 ] ; v a r b = d [ i + 2 ] ; v a r a v g = 0 . 2 1 2 6 * r + 0 . 7 1 5 2 * g + 0 . 0 7 2 2 * b ; d [ i ] = d [ i + 1 ] = d [ i + 2 ] = a v g ; } 22
23. Faster Canvas (Wish List) • Hardware acceleration • Faster GPU • 32-bit pixel manipulation (Uint8ClampedArray vs. CanvasPixelArray) • Background operation (Worker) 23
24. canvas.toBlob Converting the filtered photo (canvas) to blob i f ( c a n v a s . t o B l o b ) { v a r b l o b = c a n v a s . t o B l o b ( f u n c t i o n ( b l o b ) { / / a s y n c c a l l b a c k } , ' i m a g e / j p e g ' ) ; } 18 24
25. Blob URLs Displaying Blob < i m g s r c = " b l o b : 9 5 8 c 1 b 5 0 - 0 9 a 0 - 2 5 4 3 - 8 5 2 8 - 8 6 9 7 a e 3 8 7 6 6 7 " > v a r U R L = w i n d o w . U R L | | w i n d o w . w e b k i t U R L * ; v a r i m g U R L = U R L . c r e a t e O b j e c t U R L ( b l o b ) ; 18* 23 15bl 4* 6* 10* 10 25
26. IndexedDB Storing the photos locally i f ( w i n d o w . i n d e x e d D B ) { v a r r e q = i n d e x e d D B . o p e n ( ' c o r e m o b C a m e r a ' ) ; r e q . o n s u c c e s s = f u n c t i o n ( e ) { / / a s y n c } } 26
27. IndexedDB Chrome 18 and BlackBerry requires the webkit prefix. v a r i n d e x e d D B = w i n d o w . i n d e x e d D B | | w i n d o w . w e b k i t I n d e x e d D B ; "Basic" Async iDB Support 18* 25 15bl 10* 16 10 27
28. IndexedDB: Spotty Browser Support • Safari and Opera (presto) do not support at all • Chrome <24, Firefox <13, and BB 10 use deprecated specs • s e t V e r s i o n ( ) vs. o n u p g r a d e n e e d e d • I D B T r a n s a c t i o n . R E A D _ W R I T E vs. " r e a d w r i t e " • Chrome & Opera (blink) does not support b l o b 28
29. XMLHttpRequest Level 2 Sending a photo v a r f o r m D a t a = n e w F o r m D a t a ( ) ; f o r m D a t a . a p p e n d ( ' p h o t o ' , b l o b ) ; 3 5 10 12 15bl 29
30. XMLHttpRequest Level 2 Sending a photo v a r x h r = n e w X M L H t t p R e q u e s t ( ) ; x h r . o p e n ( ' P O S T ' , ' / g a l l e r y ' ) ; x h r . u p l o a d . a d d E v e n t L i s t e n e r ( ' p r o g r e s s ' , h a n d l e P r o g r e s s , f a l s e ) ; x h r . a d d E v e n t L i s t e n e r ( ' l o a d ' , h a n d l e F i n i s h , f a l s e ) ; x h r . a d d E v e n t L i s t e n e r ( ' e r r o r ' , h a n d l e E r r o r , f a l s e ) ; x h r . a d d E v e n t L i s t e n e r ( ' a b o r t ' , h a n d l e A b o r t , f a l s e ) ; x h r . s e n d ( f o r m D a t a ) ; 30
31. XMLHttpRequest Level 2 Sending a photo f u n c t i o n h a n d l e P r o g r e s s ( e ) { i f ( e . l e n g t h C o m p u t a b l e ) { e l . t e x t C o n t e n t = ( e . l o a d e d / e . t o t a l * 1 0 0 ) > > > 0 + ' % ' ; } } 31
32. CORS: Cross Origin Resource Sharing A c c e s s - C o n t r o l - A l l o w - O r i g i n : h t t p : / / s o m e - d o m a i n . o r g / / A p a c h e . h t a c c e s s < I f M o d u l e m o d _ h e a d e r s . c > H e a d e r s e t A c c e s s - C o n t r o l - A l l o w - O r i g i n " * " < / I f M o d u l e > 12 15bl 32
33. Touch Events v.1 Photo Gallery Carousel e l . a d d E v e n t L i s t e n e r ( ' t o u c h s t a r t ' , s t a r t H a n d l e r , f a l s e ) ; e l . a d d E v e n t L i s t e n e r ( ' t o u c h m o v e ' . . . ) ; e l . a d d E v e n t L i s t e n e r ( ' t o u c h e n d ' . . . ) ; You probably want to include mouse events too ( m o u s e d o w n, m o u s e m o v e, and m o u s e u p). 33
34. Pointer Events For any input devices: touch, mouse, pen... i f ( n a v i g a t o r . p o i n t e r E n a b l e d ) { e l . a d d E v e n t L i s t e n e r ( ' p o i n t e r d o w n ' , s t a r t H a n d l e r , f a l s e ) ; e l . a d d E v e n t L i s t e n e r ( ' p o i n t e r m o v e ' , . . . ) ; e l . a d d E v e n t L i s t e n e r ( ' p o i n t e r u p ' , . . . ) ; } 34
35. Pointer Events Only IE10 supports so far... i f ( n a v i g a t o r . m s P o i n t e r E n a b l e d ) { e l . a d d E v e n t L i s t e n e r ( ' M S P o i n t e r D o w n ' , s t a r t H a n d l e r , f a l s e ) ; e l . a d d E v e n t L i s t e n e r ( ' M S P o i n t e r M o v e ' , m o v e H a n d l e r , f a l s e ) ; e l . a d d E v e n t L i s t e n e r ( ' M S P o i n t e r U p ' , e n d H a n d l e r , f a l s e ) ; } 10 ** Experimental Chromium build (26.0.1409.0) - no vendor prefix 35
36. More... • History API ( p o p s t a t e event) • Screen Orientation API ( o r i e n t a t i o n c h a n g e event) • CSS3 Transforms and Animations • Web Fonts ( @ f o n t - f a c e) • h i d d e n attribute • d a t a - * custom attribute 36
37. PhoneGap HTML5 + Native Filling the gap of missing HTML5 features 37
38. github.com/coremob/camera  38
39. coremob.github.io/camera/vanilla/index.html goo.gl/My8mf 39
40. Thank you! Tomomi Imura • girliemac.com • twitter.com/girlie_mac • github.com/girliemac • github.com/coremob/camera 40