Automating Business with Google

Automating Business with Google Apps including Drive and Spreadsheet as well as Google App Engine.

Transcript

1. Automating Business with Google APIs David Hodge Thursday, March 7, 13
2. Thursday, March 7, 13
3. Thursday, March 7, 13
4. Thursday, March 7, 13
5. Thursday, March 7, 13
6. We automate business processes using cloud computing and machine learning. Dave & Bediako Thursday, March 7, 13
7. IAAS and PAAS - Amazon, Google, Microsoft - DevOps, NoOps, OOPs - Beware single point of failure - read SLAs Thursday, March 7, 13
8. Google Docs Drive • Google Docs API deprecated • Google Drive Polyglot API - .NET, Java, Dart, Go, GWT, Java, Javascript, Objective- C, PHP, Python, Ruby • ReSTful API - create and manage documents Thursday, March 7, 13
9. Google Spreadsheets API • Spreadsheets, Worksheets, List Row, Cell • Create, Delete, Share • List Based Feeds vs Cell Based Feeds • Updating multiple cells with Batch Thursday, March 7, 13
10. Google Spreadsheets API • Gotchas • Queries - age > 25 and height < 175 Thursday, March 7, 13
11. Google Apps Scripts • Javascript cloud scripting language • Automate workflows • Customize spreadsheets • Schedule Tasks • Save data using Spreadsheets, JDBC, ScriptDB Thursday, March 7, 13
12. Google App Engine Thursday, March 7, 13
13. Google App Engine Backends Thursday, March 7, 13
14. Google App Engine Backends Storing Data Thursday, March 7, 13
15. Google App Engine Backends Storing Data Blobstore Thursday, March 7, 13
16. Google App Engine Backends Storing Data Blobstore App Identity Thursday, March 7, 13
17. Google App Engine Backends Storing Data Blobstore App Identity Google Cloud Storage Thursday, March 7, 13
18. Google App Engine Backends Storing Data Blobstore App Identity Google Cloud Storage Capabilities Thursday, March 7, 13
19. Google App Engine Backends Storing Data Blobstore App Identity Google Cloud Storage Capabilities Channel Thursday, March 7, 13
20. Google App Engine Backends Storing Data Blobstore App Identity Google Cloud Storage Capabilities Channel Endpoints Thursday, March 7, 13
21. Google App Engine Backends Storing Data Blobstore App Identity Google Cloud Storage Capabilities Channel Endpoints Images Thursday, March 7, 13
22. Google App Engine Backends Storing Data Blobstore App Identity Google Cloud Storage Capabilities Channel Endpoints Images Logs Thursday, March 7, 13
23. Google App Engine Backends Storing Data Blobstore App Identity Google Cloud Storage Capabilities Channel Endpoints Images Logs Mail Thursday, March 7, 13
24. Google App Engine Backends Storing Data Blobstore App Identity Google Cloud Storage Capabilities Channel Endpoints Images Logs Mail Memcache Thursday, March 7, 13
25. Google App Engine Backends Storing Data Blobstore App Identity Google Cloud Storage Capabilities Channel Endpoints Images Logs Mail Memcache Multitenancy Thursday, March 7, 13
26. Google App Engine Backends Storing Data Blobstore App Identity Google Cloud Storage Capabilities Channel Endpoints Images Logs Mail Memcache Multitenancy OAuth Thursday, March 7, 13
27. Google App Engine Backends Storing Data Blobstore App Identity Google Cloud Storage Capabilities Channel Endpoints Images Logs Mail Memcache Multitenancy OAuth Prospective Search Thursday, March 7, 13
28. Google App Engine Backends Storing Data Blobstore App Identity Google Cloud Storage Capabilities Channel Endpoints Images Logs Mail Memcache Multitenancy OAuth Prospective Search Search Thursday, March 7, 13
29. Google App Engine Backends Storing Data Blobstore App Identity Google Cloud Storage Capabilities Channel Endpoints Images Logs Mail Memcache Multitenancy OAuth Prospective Search Search Task Queues Thursday, March 7, 13
30. Google App Engine Backends Storing Data Blobstore App Identity Google Cloud Storage Capabilities Channel Endpoints Images Logs Mail Memcache Multitenancy OAuth Prospective Search Search Task Queues Users Thursday, March 7, 13
31. Google App Engine Backends Storing Data Blobstore App Identity Google Cloud Storage Capabilities Channel Endpoints Images Logs Mail Memcache Multitenancy OAuth Prospective Search Search Task Queues Users XMPP Thursday, March 7, 13
32. One does not simply Automate using Google APIs Thursday, March 7, 13
33. How we build software A Customer has a nick name has a full name has a date of birth has a social security number which is sensitive can be viewed or created or changed by a customer service representative An Order belongs to a Customer has a description can be viewed or created or changed by a customer service representative An Order Line Item is audited belongs to an Order has a description has a price which is a number can be viewed or created or changed by a customer service representative Dictation + Airlift on Google App Engine Thursday, March 7, 13
34. Current processes manual and paper based for recording line items for accounts receivable and accounts payable. Thursday, March 7, 13
35. Customer Widgets Spreadsheets Thursday, March 7, 13
36. Widget Maker Spreadsheets Thursday, March 7, 13
37. Evolution One - Cron <?xml version="1.0" encoding="UTF-8"?> <cronentries> <cron> <url>/accounting/receivable</url> <description>Process widgets sold for accounting line items</description> <schedule>every day 12:30</schedule> <timezone>America/New_York</timezone> </cron> <cron> <url>/accounting/payable</url> <description>Process line items for widget making workers</description> <schedule>every day 03:30</schedule> <timezone>America/New_York</timezone> </cron> </cronentries> Thursday, March 7, 13
38. Evolution Two - Backends • Use Backends to process long lived requests - No Request Deadlines • Higher CPU up to 4.8GHz and Memory limits - up to 1GB • Also using Spreadsheet Batch Thursday, March 7, 13
39. Evolution Two - Backends • Dynamic vs Resident - Dynamic Backends become alive only on request and are turned off when idle Thursday, March 7, 13
40. Evolution Two - Backends <backends> <backend name="accounting"> <options> <dynamic>true</dynamic> <fail-fast>true</fail-fast> <class>B8</class> </options> </backend> </backends> backends.xml <?xml version="1.0" encoding="UTF-8"?> <cronentries> <cron> <url>/accounting/receivable</url> <description>Process widgets sold for accounting line items</description> <schedule>every 20 minutes</schedule> <timezone>America/New_York</timezone> <target>accounting</target> </cron> <cron> <url>/accounting/payable</url> <description>Process line items for widget making workers</description> <schedule>every 40 minutes</schedule> <timezone>America/New_York</timezone> <target>accounting</target> </cron> </cronentries> cron.xml Thursday, March 7, 13
41. Task Queues work discretely on a unit of work Push - processes tasks based on rate configuration, auto scales Pull - allows for specific time and timeframe to run Thursday, March 7, 13
42. Task Push Config <queue-entries> <queue> <name>fooqueue</name> <rate>1/s</rate> <bucket-size>40</bucket-size> <max-concurrent-requests>1</max-concurrent-requests> <retry-parameters> <task-retry-limit>7</task-retry-limit> <task-age-limit>2</task-age-limit> </retry-parameters> </queue> <queue> <name>barqueue</name> <rate>1/s</rate> <retry-parameters> <min-backoff-seconds>10</min-backoff-seconds> <max-backoff-seconds>200</max-backoff-seconds> <max-doublings>0</max-doublings> </retry-parameters> </queue> </queue-entries> queue.xml Thursday, March 7, 13
43. Task Queue Headers • X-AppEngine-QueueName, the name of the queue (possibly default) • X-AppEngine-TaskName, the name of the task, or a system-generated unique ID if no name was specified • X-AppEngine-TaskRetryCount, the number of times this task has been retried; for the first attempt, this value is 0. This number includes attempts where the task failed due to a lack of available instances and never reached the execution phase. • X-AppEngine-TaskExecutionCount, the number of times this task has previously failed during the execution phase. This number does not include failures due to a lack of available instances. • X-AppEngine-TaskETA, the target execution time of the task, specified in microseconds since January 1st 1970. Thursday, March 7, 13
44. Task Push Queues import com.google.appengine.api.taskqueue.Queue; import com.google.appengine.api.taskqueue.QueueFactory; import static com.google.appengine.api.taskqueue.TaskOptions.Builder.*; //Discrete job for processing single widget to write customer line item Queue queue = QueueFactory.getDefaultQueue(); queue.add(withUrl("/widgetorderforcustomerjob").param("widgetKey", key)); //Discrete job for processing single widget to write vendor line item queue.add(withUrl("/widgetorderforvendorjob").param("widgetKey", key)); Thursday, March 7, 13
45. Google Docs Google App Engine Automation Achieved Receivable Line Items Job Payable Line Items Job Google Spreadsheet Line Items Per Client Google Spreadsheet Line Items Per Vendor Invoices Payments Thursday, March 7, 13
46. Experiment Evolve Architecture Thursday, March 7, 13
47. Support • Google Plus Hangouts • Google Forums • Fellow Developers Thursday, March 7, 13
48. Some More Tips • Python: Bulk Loader Tool to download/ upload data - good for any flavor of GAE • Get involved - NoOps doesn’t mean you don’t have to know your technology stack • https://github.com/LucidTechnics/google- apps-util • https://code.google.com/p/googleappengine/ wiki/WillItPlayInJava Thursday, March 7, 13
49. “Using no way as a way, having no limitation as limitation.” - Bruce Lee Thursday, March 7, 13
50. Official White House Photo by Pete Souza Thursday, March 7, 13
51. David Hodge @davezen1 Thursday, March 7, 13