Intro to Google Cloud Datastore - #GDGSV

Slide 1

Slide 1 text

Google Cloud Datastore Sandeep Dinesh, Developer Advocate @sandeepdinesh

Slide 2

Slide 2 text

● Developer Advocate @ Google ● Full-Stack Web ● Founded an IoT Startup ● First talk at Google! ● @sandeepdinesh github.com/thesandlord [email protected] About me

Slide 3

Slide 3 text

Datastore, NoSQL, RDBMS, Graphs, ... What should I use? How are they different?

Slide 4

Slide 4 text

How about SQL?

Slide 5

Slide 5 text

● Relational Data ● Complex Queries ● Track Record The old guard RDBMS - Relational Databases Images from Wikipedia

Slide 6

Slide 6 text

● Scale ● Flexibility The old guard RDBMS - Relational Databases Images from Wikipedia

Slide 7

Slide 7 text

The new kids on the block NoSQL

Slide 8

Slide 8 text

The new kids on the block NoSQL NoSQL is a general term

Slide 9

Slide 9 text

Document Store The new kids on the block NoSQL NoSQL is a general term

Slide 10

Slide 10 text

Document Store Graph/Triplestore The new kids on the block NoSQL NoSQL is a general term

Slide 11

Slide 11 text

Document Store Graph/Triplestore Key-Value The new kids on the block NoSQL NoSQL is a general term

Slide 12

Slide 12 text

Document Store Graph/Triplestore Key-Value The “new” kids on the block NoSQL NoSQL is a general term Distributed

Slide 13

Slide 13 text

What is Datastore? And why should I care...

Slide 14

Slide 14 text

A little background...

Slide 15

Slide 15 text

Google App Engine Google Datastore

Slide 16

Slide 16 text

Scale Performance Availability Sharding Replication ???

Slide 17

Slide 17 text

Fully Managed NoSQL ACID Built for Scale High Replication High Availability Google Cloud Datastore

Slide 18

Slide 18 text

Fully Managed NoSQL ACID Built for Scale High Replication High Availability Google Cloud Datastore Google’s DBAs are your DBAs No software upgrades/patches No pager duty :)

Slide 19

Slide 19 text

Fully Managed NoSQL ACID Built for Scale High Replication High Availability Google Cloud Datastore Semi-Structured Data Easy to Use Powerful Query Abilities

Slide 20

Slide 20 text

Fully Managed NoSQL ACID Built for Scale High Replication High Availability Google Cloud Datastore Atomic Transactions (Eventual) Consistency Reliable

Slide 21

Slide 21 text

Fully Managed NoSQL ACID Built for Scale High Replication High Availability Google Cloud Datastore Built for Google Scale Scalable Design, By Design

Slide 22

Slide 22 text

Fully Managed NoSQL ACID Built for Scale High Replication High Availability Google Cloud Datastore Asteroid Proof 99.95% uptime

Slide 23

Slide 23 text

Section Slide Template Option 2 Put your subtitle here. Feel free to pick from the handful of pretty Google colors available to you. Make the subtitle something clever. People will think it’s neat. Using Datastore Getting Started with Infinite Scale

Slide 24

Slide 24 text

Datastore Introduction Terminology Datastore RDBMS Category of Object Kind Table One entry/object Entity Row Unique identifier of data entry Key Primary Key Individual piece of data inside entry/object Property Field

Slide 25

Slide 25 text

Datastore Introduction Terminology Employees Employee Employees Employee Employee Employee Employee Employee ID: 123456 Name Address Title

Slide 26

Slide 26 text

Datastore Introduction Terminology Eventual Consistency vs Strong Consistency

Slide 27

Slide 27 text

Terminology Eventual Consistency Data Center 1 Data Center 2 User 1 New Data User 2 User 3 Old Data New Data New Data

Slide 28

Slide 28 text

Terminology Strong Consistency Data Center 1 Data Center 2 User 1 New Data User 2 User 3 Locked Locked New Data New Data

Slide 29

Slide 29 text

Terminology Strong Consistency Entity Groups & Ancestor Keys

Slide 30

Slide 30 text

Terminology Eventual Consistency vs Strong Consistency Common Types of Queries: Global Queries Key Queries Ancestor Queries Eventually Consistent Strongly Consistent Strongly Consistent

Slide 31

Slide 31 text

Code Yay!

Slide 32

Slide 32 text

Code APIs JSON & Protobuf

Slide 33

Slide 33 text

Code The gcloud library Idiomatic and Open Source

Slide 34

Slide 34 text

Code Scenario Let’s build a message board Full Code w/ ncurses UI: github.com/thesandlord/examples/datastore/ All code I am going to write is Copyright Google and Licensed under Apache v2

Slide 35

Slide 35 text

Code Setup Terminal $ pip install --upgrade gcloud $ pip install --upgrade google-api-python-client

Slide 36

Slide 36 text

Code Setup

Slide 37

Slide 37 text

Code Architecture Comment ID, Text, StoryID User ID, Name Ancestor ID, Name, URL, Text Stories

Slide 38

Slide 38 text

Code Oh no it broke :( Query with Multiple Filters need special Indexes! query = datastore.Query(kind='Comment', filters=[('storyid', '=', obj["id"])], order=['timestamp']) gcloud.exceptions.PreconditionFailed: 412 no matching index found.

Slide 39

Slide 39 text

Code Create Some Indexes! We need to use the gcd tool https://cloud.google.com/datastore/docs/downloads#tools gcd.sh create ./temp mv temp/WEB-INF . rm -r temp/ #Create Index XML File (next slide) gcd.sh updateindexes --auth_mode=oauth2 . Follow the prompts

Slide 40

Slide 40 text

Code datastore-indexes.xml XML

Slide 41

Slide 41 text

Code Index Created!

Slide 42

Slide 42 text

Thank you! Thank you! twitter.com/sandeepdinesh github.com/thesandlord/examples/datastore/