Google Cloud Endpoints and RESTful APIs

This is a slide deck of a talk given to a UCL/Birkbeck students in May 2013.
It covers following topics:

* What are RESTful APIs and what are they useful for
* Building RESTful APIs using Cloud Endpoints and Google AppEngine in Python
* Building Javascript clients for these APIs

Transcript

2. Google Cloud Endpoints Third-party APIs on Google AppEngine Roman Kirillov Software Engineer, Google
3. Hello World Some introductions are in order Google Cloud Endpoints My name is Roman. "Mr Kirillov" makes me look around. If you have a question, raise a hand. There will be dedicated Q&A time in the end of the lecture All source code will be provided in form of a GitHub link · · · · 3/39
4. Today's plan Google Cloud Endpoints What is APIs, what are Web APIs and why do we need them? What is the architecture and structure of a Web API How to build APIs using Google AppEngine How to consume APIs on the web and mobile devices · · · · 4/39
5. What is API and why do we need them?
6. One simple question What is cloud computing? “ ”
7. Example: Astrid Tasks Google Cloud Endpoints 7/39
8. What is API An application programming interface (API) is a protocol intended to be used as an interface by software components to communicate with each other. Google Cloud Endpoints “ ” OK, it's really Wikipedia, but never mind — Application Programming Interface 8/39
9. What is API ...now the same, but in smaller words API is: Widely known examples: Google Cloud Endpoints A layer between two applications Is all about communication Well defined interface, which hides implementation · · · Java Collections API OpenGL – cross-platform graphics API Hadoop API · · · 9/39
10. Web APIs Why are they different
11. Web APIs Very simple example: Astrid Tasks API: Google Cloud Endpoints Work over HTTP or HTTPS Normally are stateless · · P O S T h t t p s : / / a s t r i d . c o m / a p i / 7 / t a s k _ l i s t D a t a : " t o k e n : 1 2 3 4 5 " Data is POSTed to some URL (not only POST can be used) Request and response in JSON or XML format · · Can't pass just programming language-level objects via HTTP. · 11/39
12. Restful APIs Astrid API and others of that type are often called RESTful APIs. My goal today is to show you how you can build one. Google Cloud Endpoints REST = REpresentational State Transfer Use HTTP POST, PUT, DELETE and GET methods Expose hierarchical data Stateless! (usually available over HTTP[S]) · · · · · 12/39
13. Restful APIs vs just Web APIs What's the difference, really? Google Cloud Endpoints How I Explained REST to My Wife – read it. Seriously. Restful APIs are Web services which use HTTP the right way HTTP GET – listing entities, GET never used to modify the data HTTP POST – adding entity (insert) HTTP PUT – updating HTTP DELETE – deletion · · P O S T / u s e r ? i d = 1 2 3 4 & n a m e = r o m a n v s P U T / u s e r / 1 2 3 4 : { n a m e : " r o m a n " } G E T / u s e r _ d e l e t e ? i d = 1 2 3 4 & u s e r _ g r o u p = 1 v s D E L E T E / u s e r s / 1 / 1 2 3 4 · · · · · 13/39
14. Restful APIs Principal data flow diagram Web Server Request Convert to Object Process request Data store Convert to Message Web Server Request Boilerplate! We don't want to do this every time! Business logic. We want to concentrate on this! Google Cloud Endpoints 14/39
15. Building Restful APIs – normal steps Google Cloud Endpoints can make your life substantially more simple. Google Cloud Endpoints Define logical model of the data Define a mapping between logical data model and JSON, XML; define a mapping between database (SQL) and logical model Build a converter from JSON (XML) to programming language objects Build a converter – often very non-trivial – from programming language objects to SQL database Configure a web server capable of receiving GET/POST/PUT/DELETE requests Configure mapping between scripts/servlets and URLs ... now you can actually start implementing your service We haven't even started talking about security, authentication, API versioning and supporting multiple client platforms · · · · · · · · 15/39
16. Let's build something!
17. Google Cloud Endpoints Google Cloud Endpoints source: developers.google.com 17/39
18. Sensors API Here's a real-world problem for us to solve: How will we build it? Google Cloud Endpoints Home automation system, consists of many sensors; Sensors can have different type; Sensors report to home server; We want remotely enable and disable sensors; Web and mobile clients are required. · · · · · 18/39
19. Google Cloud Endpoints Google Cloud Endpoints 19/39
20. Building API with Cloud Endpoints To build an API on Google AppEngine we need: Let's go through these steps. Google Cloud Endpoints Define the [data] model Define web service interface Implement conversion between RPC and datastore layers Implement the business logic of the API · · · · 20/39
21. Building the API Defining API model AppEngine Datastore class to persist our sensors Google Cloud Endpoints # R e p r e s e n t s a s i n g l e s e n s o r i n a p p l i c a t i o n ' s d a t a b a s e c l a s s S e n s o r ( d b . M o d e l ) : s e n s o r _ i d = d b . I n t e g e r P r o p e r t y ( r e q u i r e d = T r u e ) n e t w o r k _ i d = d b . S t r i n g P r o p e r t y ( ) r o o m = d b . S t r i n g P r o p e r t y ( ) s e n s o r _ t y p e = d b . S t r i n g P r o p e r t y ( ) a c t i v e = d b . B o o l e a n P r o p e r t y ( ) P Y T H O N 21/39
22. Building the API Defining a message in ProtoRPC format Google Cloud Endpoints c l a s s S e n s o r M e s s a g e ( m e s s a g e s . M e s s a g e ) : c l a s s R o o m ( m e s s a g e s . E n u m ) : L I V I N G _ R O O M = 1 B E D R O O M = 2 c l a s s T y p e ( m e s s a g e s . E n u m ) : T E M P E R A T U R E = 1 M O T I O N = 2 s e n s o r _ i d = m e s s a g e s . I n t e g e r F i e l d ( 1 , r e q u i r e d = T r u e ) n e t w o r k _ i d = m e s s a g e s . S t r i n g F i e l d ( 2 ) r o o m = m e s s a g e s . E n u m F i e l d ( ' S e n s o r M e s s a g e . R o o m ' , 3 , d e f a u l t = ' L I V I N G _ R O O M ' ) s e n s o r _ t y p e = m e s s a g e s . E n u m F i e l d ( ' S e n s o r M e s s a g e . T y p e ' , 4 , d e f a u l t = ' T E M P E R A T U R E ' ) a c t i v e = m e s s a g e s . B o o l e a n F i e l d ( 5 , d e f a u l t = T r u e ) P Y T H O N 22/39
23. Building the API RPC ↔ datastore conversion Fairly trivial converter. Will work in other direction similarly. Can probably be generalised for other types of messages. Google Cloud Endpoints d e f S e n s o r R p c T o D b ( s ) : " " " C o n v e r t s S e n s o r M e s s a g e t o d b - l e v e l S e n s o r o b j e c t . A r g s : s : S e n s o r M e s s a g e R e t u r n s : I n s t a n c e o f S e n s o r . " " " r e t u r n S e n s o r ( k e y _ n a m e = s t r ( s . s e n s o r _ i d ) , s e n s o r _ i d = s . s e n s o r _ i d , n e t w o r k _ i d = s . n e t w o r k _ i d , r o o m = s t r ( s . r o o m ) , s e n s o r _ t y p e = s t r ( s . s e n s o r _ t y p e ) , a c t i v e = s . a c t i v e ) P Y T H O N 23/39
24. Example: Insert method Adding a new sensor to the datastore Google Cloud Endpoints d e f s e n s o r s _ i n s e r t ( s e l f , r e q u e s t ) : r e s p o n s e = S e n s o r s R e s p o n s e M e s s a g e ( ) r e s p o n s e . s t a t u s = S t a t u s . S T A T U S _ O K e r r o r s = [ ] f o r s i n r e q u e s t . s e n s o r s : i f S e n s o r . g e t _ b y _ k e y _ n a m e ( s t r ( s . s e n s o r _ i d ) ) : r e s p o n s e . s t a t u s = S t a t u s . S T A T U S _ F A I L E D e r r o r s . a p p e n d ( " s e n s o r _ i d = " + s . s e n s o r _ i d + " e x i s t s " ) e l s e : s e n s o r E n t i t y = S e n s o r R p c T o D b ( s ) s e n s o r E n t i t y . p u t ( ) r e s p o n s e . e r r o r = e r r o r s r e t u r n r e s p o n s e P Y T H O N 24/39
25. Wiring it all together Annotating main API class: Annotating API methods: Google Cloud Endpoints @ e n d p o i n t s . a p i ( n a m e = ' s e n s o r s ' , v e r s i o n = ' v 1 ' , d e s c r i p t i o n = ' S e n s o r s A P I ' , a l l o w e d _ c l i e n t _ i d s = [ C L I E N T _ I D , e n d p o i n t s . A P I _ E X P L O R E R _ C L I E N T _ I D ] ) c l a s s S e n s o r s A p i ( r e m o t e . S e r v i c e ) : # . . . P Y T H O N # . . . c l a s s S e n s o r s A p i ( r e m o t e . S e r v i c e ) : @ e n d p o i n t s . m e t h o d ( S e n s o r s R e q u e s t M e s s a g e , S e n s o r s R e s p o n s e M e s s a g e , p a t h = ' s e n s o r s ' , h t t p _ m e t h o d = ' P O S T ' , n a m e = ' s e n s o r s . i n s e r t ' ) d e f s e n s o r s _ i n s e r t ( s e l f , r e q u e s t ) : # . . . P Y T H O N 25/39
26. Let's get it started Google Cloud Endpoints Google AppEngine Launcher with running app 26/39
27. Google API Explorer Useful tool for exploring and testing APIs Google Cloud Endpoints https://developers.google.com/apis-explorer/?base=http://localhost:8080/_ah/api#p/ 27/39
28. Google API Explorer: methods Google Cloud Endpoints 28/39
29. Google API Explorer: Insert Google Cloud Endpoints 29/39
30. Request and response Google Cloud Endpoints P O S T h t t p : / / l o c a l h o s t : 8 0 8 0 / _ a h / a p i / s e n s o r s / v 1 / s e n s o r s C o n t e n t - T y p e : a p p l i c a t i o n / j s o n X - J a v a S c r i p t - U s e r - A g e n t : G o o g l e A P I s E x p l o r e r { " s e n s o r s " : [ { " a c t i v e " : f a l s e , " n e t w o r k _ i d " : " 1 9 2 . 1 6 8 . 0 . 1 4 " , " r o o m " : " L I V I N G _ R O O M " , " s e n s o r _ i d " : " 1 " , " s e n s o r _ t y p e " : " T E M P E R A T U R E " } ] } 2 0 0 O K C o n t e n t - L e n g t h : 2 3 # s o m e h e a d e r s o m i t t e d f o r b r e v i t y – R K S e r v e r : D e v e l o p m e n t / 1 . 0 { " s t a t u s " : " S T A T U S _ O K " } 30/39
31. Calling API from a command line Quick WTF: Google Cloud Endpoints [ k i r i l l o v @ % M a c B o o k A i r : ~ ] $ c u r l \ - - h e a d e r " C o n t e n t - T y p e : a p p l i c a t i o n / j s o n " - X P O S T \ - d ' { " s e n s o r s " : [ { \ " a c t i v e " : f a l s e , \ " n e t w o r k _ i d " : " 1 9 2 . 1 6 8 . 0 . 1 6 " , \ " r o o m " : " L I V I N G _ R O O M " , \ " s e n s o r _ i d " : " 2 " , \ " s e n s o r _ t y p e " : " T E M P E R A T U R E " } ] } ' \ h t t p : / / l o c a l h o s t : 8 0 8 0 / _ a h / a p i / s e n s o r s / v 1 / s e n s o r s { " s t a t u s " : " S T A T U S _ O K " } % - X P O S T specifies request type - - h e a d e r " C o n t e n t - T y p e : a p p l i c a t i o n / j s o n " specifies type of data - d specifies the actual content we are POSTing l o c a l h o s t : 8 0 8 0 / _ a h / a p i / s e n s o r s / v 1 / s e n s o r s is the URL we are POSTing to · · · · 31/39
32. Web Client Add to your HTML < h e a d > section Create JavaScript i n i t ( ) function Google Cloud Endpoints < s c r i p t s r c = " h t t p s : / / a p i s . g o o g l e . c o m / j s / c l i e n t . j s ? o n l o a d = i n i t " > < / s c r i p t > f u n c t i o n i n i t ( ) { v a r R O O T = ' h t t p : / / l o c a l h o s t : 8 0 8 0 / _ a h / a p i ' ; g a p i . c l i e n t . l o a d ( ' s e n s o r s ' , ' v 1 ' , f u n c t i o n ( ) { / / j u s t a n e x a m p l e l i s t S e n s o r s ( ) ; } , R O O T ) ; } J S Replace R O O T with actual API root URL Start using the API · · 32/39
33. Example: adding new sensor Google Cloud Endpoints f u n c t i o n c r e a t e S e n s o r ( ) { v a r s e n s o r = { / / g e n e r a t e r a n d o m l y ' s e n s o r _ i d ' : r a n d ( 7 8 9 4 7 5 6 4 ) , ' n e t w o r k _ i d ' : ' 1 9 2 . 1 6 8 . 0 . ' + r a n d ( 2 5 4 ) , ' r o o m ' : r a n d ( [ ' L I V I N G _ R O O M ' , ' B E D R O O M ' , ' H A L L ' ] ) , ' s e n s o r _ t y p e ' : r a n d ( [ ' T E M P E R A T U R E ' , ' M O T I O N ' ] ) , ' a c t i v e ' : r a n d ( [ t r u e , f a l s e ] ) } ; v a r r e q u e s t = { ' s e n s o r s ' : [ s e n s o r ] } ; g a p i . c l i e n t . s e n s o r s . s e n s o r s . i n s e r t ( r e q u e s t ) . e x e c u t e ( f u n c t i o n ( r e s p ) { l i s t S e n s o r s ( ) ; } ) ; } J S 33/39
34. Example: listing sensors List method returns SensorResponseMessage, which in turn contains a collection of SensorMessage objects. Google Cloud Endpoints f u n c t i o n l i s t S e n s o r s ( ) { g a p i . c l i e n t . s e n s o r s . s e n s o r s . l i s t ( ) . e x e c u t e ( f u n c t i o n ( r e s p ) { / / t h i s i s s o m e w h a t p s e u d o - c o d e - i s h b u t w i l l d o / / f o r t h e p u r p o s e o f t h i s e x a m p l e . H T M L i s u g l y . v a r t a b l e = c r e a t e T a b l e ( ) ; r e s p . s e n s o r s . f o r E a c h ( f u n c t i o n ( s e n s o r ) { t a b l e . a d d S e n s o r ( s e n s o r . s e n s o r _ i d , s e n s o r . n e t w o r k _ i d , s e n s o r . s e n s o r _ t y p e , s e n s o r . r o o m , s e n s o r . a c t i v e ) ; } ) ; } ) ; } J S 34/39
35. Example: Toggle Active Status Update method takes SensorMessage object. Google Cloud Endpoints f u n c t i o n t o g g l e A c t i v e ( s e n s o r I d , a c t i v e ) { v a r r e q u e s t = { ' s e n s o r _ i d ' : s e n s o r I d , ' a c t i v e ' : a c t i v e } ; g a p i . c l i e n t . s e n s o r s . s e n s o r s . u p d a t e ( r e q u e s t ) . e x e c u t e ( f u n c t i o n ( r e s p ) { l i s t S e n s o r s ( ) ; } ) ; } J S 35/39
36. Example: making it click Google Cloud Endpoints 0:20 36/39
37. Other platforms Cloud Endpoints will also work with: Google Cloud Endpoints Android – native library generator iOS – native library generator Python/PHP/Ruby – via Google API library Every single platform capable of making HTTP requests · · · · 37/39
38. <Thank You!> g+ google.com/+RomanKirillov twitter @sgzmd github github.com/sigizmund