Tweet
Tweet

Slide 1

Slide 1 text

‹#› @tpryan Terrence Ryan Developer Advocate 30,000,000 Requests An Hour In The Cloud

Slide 2

Slide 2 text

‹#› @tpryan Who are you?

Slide 3

Slide 3 text

‹#› @tpryan 01 Introduction What was I trying to do

Slide 4

Slide 4 text

No content

Slide 5

Slide 5 text

‹#› @tpryan Demo: App Engine

Slide 6

Slide 6 text

‹#› @tpryan App Engine • Writes to Firebase • Writes to Cloud Storage • Run the Visualizer

Slide 7

Slide 7 text

‹#› @tpryan Compute Engine • Load Generators • Apache Bench • Remote Command Execution

Slide 8

Slide 8 text

‹#› @tpryan Firebase • Realtime Display • Tracking all load

Slide 9

Slide 9 text

‹#› @tpryan Cloud Storage • Proof of activity • Object notification

Slide 10

Slide 10 text

‹#› @tpryan App Engine • Process Notifications from Cloud Storage

Slide 11

Slide 11 text

‹#› @tpryan Planned / //Write to Firebase //Write to Cloud Storage //Receive realtime
 //updates /visualizer /storage //Write to Firebase //Display updates //Store file //Send Notification http://load-demo.appspot.com

Slide 12

Slide 12 text

‹#› @tpryan Actual / //Write to Firebase //Write to Cloud Storage //Receive realtime
 //updates /visualizer /storage //Write to Firebase //Display updates //Store file //Send Notification http://load-demo.appspot.com

Slide 13

Slide 13 text

@briandorsey 30,000,000 requests in one hour

Slide 14

Slide 14 text

@briandorsey What do you call a site that gets 30,000,000 hits an hour?

Slide 15

Slide 15 text

@briandorsey

Slide 16

Slide 16 text

@briandorsey I should try and do that and mean it

Slide 17

Slide 17 text

‹#› @tpryan 02 Ramping up What is Google Cloud Platform

Slide 18

Slide 18 text

@briandorsey 30,000,000 qph /60/60 8,333 qps

Slide 19

Slide 19 text

@briandorsey Ramp the demo up to 8,333 qps

Slide 20

Slide 20 text

‹#› @tpryan Demo: App Engine

Slide 21

Slide 21 text

‹#› @tpryan URLFetch • Network abstraction layer for http • Most service calls

Slide 22

Slide 22 text

‹#› @tpryan Source Code $instance = $_SERVER['INSTANCE_ID']; $urlToPatch = $fbBaseURL . "appengine/" . $instance . ".json"; $urlToPost = $fbBaseURL . "appengine/" . $instance . “/sessions.json"; $instance_data = new stdClass(); $instance_data->instance = $instance; $instance_data->updated = time(); $instance_json = json_encode($instance_data); $ch = curl_init(); $output = patch($ch, $urlToPatch, $instance_json); $output = post($ch, $urlToPost, $instance_json); $name = json_decode($output)->name; $filename = $gcsBaseURL . $instance . "/file" . $name . ".txt"; file_put_contents($filename, $name); $instance = $_SERVER['INSTANCE_ID']; $urlToPatch = $fbBaseURL . "appengine/" . $instance . ".json"; $urlToPost = $fbBaseURL . "appengine/" . $instance . “/sessions.json"; $instance_data = new stdClass(); $instance_data->instance = $instance; $instance_data->updated = time(); $instance_json = json_encode($instance_data); $ch = curl_init(); $output = patch($ch, $urlToPatch, $instance_json); $output = post($ch, $urlToPost, $instance_json); $name = json_decode($output)->name; $filename = $gcsBaseURL . $instance . "/file" . $name . ".txt"; file_put_contents($filename, $name);

Slide 23

Slide 23 text

‹#› @tpryan Source Code function patch($ch, $url, $json){ curl_setopt($ch, CURLOPT_URL, $url); curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); curl_setopt($ch, CURLOPT_CUSTOMREQUEST, "PATCH"); curl_setopt($ch, CURLOPT_POSTFIELDS, $json); return curl_exec($ch); } function post($ch, $url, $json){ curl_setopt($ch, CURLOPT_URL, $url); curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); curl_setopt($ch, CURLOPT_CUSTOMREQUEST, "POST"); curl_setopt($ch, CURLOPT_POSTFIELDS, $json); return curl_exec($ch); }

Slide 24

Slide 24 text

‹#› @tpryan

Slide 25

Slide 25 text

‹#› @tpryan

Slide 26

Slide 26 text

‹#› @tpryan

Slide 27

Slide 27 text

@briandorsey 660,000 qpm /60 11,000 qps /3 3,666 qps

Slide 28

Slide 28 text

‹#› @tpryan 03 Fix 1 Sockets

Slide 29

Slide 29 text

‹#› @tpryan

Slide 30

Slide 30 text

‹#› @tpryan

Slide 31

Slide 31 text

‹#› @tpryan

Slide 32

Slide 32 text

‹#› @tpryan Demo: App Engine

Slide 33

Slide 33 text

‹#› @tpryan Reasons for failure • Sockets are lighter • But implementation is not configurable • So write your own socket client?

Slide 34

Slide 34 text

‹#› @tpryan 04 Fix 2 Memcache

Slide 35

Slide 35 text

‹#› @tpryan Assumptions • Visualization needs to be realtime • But does it?

Slide 36

Slide 36 text

‹#› @tpryan Memcache on App Engine

Slide 37

Slide 37 text

‹#› @tpryan Pseudo Code // store instances request count // Get list of keys // Display request counts Bad Idea

Slide 38

Slide 38 text

‹#› @tpryan

Slide 39

Slide 39 text

‹#› @tpryan Source Code $memcache = new Memcached; $instance = $_SERVER['INSTANCE_ID']; $request = $_SERVER['REQUEST_LOG_ID']; $result = $memcache->increment($instance); if ($result == 1 ){ $result = $memcache->append("instances", "|" . $instance); if ($memcache->getResultCode() == 14){ $result = $memcache->set("instances", $instance); } } $filename = $gcsBaseURL . $instance . "/file" . $request . ".txt"; file_put_contents($filename, $filename);

Slide 40

Slide 40 text

‹#› @tpryan Close but no cigar • Losing instances • Analysis revealed: • All requests were firing • All requests were registering • Cause: Memcache strings were getting too long

Slide 41

Slide 41 text

‹#› @tpryan Source Code $memcache = new Memcached; $instance = $_SERVER['INSTANCE_ID']; $request = $_SERVER['REQUEST_LOG_ID']; $memcache->increment("total"); $result = $memcache->increment($instance); if ($result == 1 ){ $result = $memcache->append("instances", "|" . $instance); if ($memcache->getResultCode() == 14){ $result = $memcache->set("instances", $instance); } } $filename = $gcsBaseURL . $instance . "/file" . $request . ".txt"; file_put_contents($filename, $filename); $memcache = new Memcached; $instance = $_SERVER['INSTANCE_ID']; $request = $_SERVER['REQUEST_LOG_ID']; $memcache->increment("total"); $result = $memcache->increment($instance); if ($result == 1 ){ $result = $memcache->append("instances", "|" . $instance); if ($memcache->getResultCode() == 14){ $result = $memcache->set("instances", $instance); } } $filename = $gcsBaseURL . $instance . "/file" . $request . ".txt"; file_put_contents($filename, $filename);

Slide 42

Slide 42 text

‹#› @tpryan Source Code function whichInstanceList($instance){ $last = substr($instance, -1); return "instances-" . $last; }

Slide 43

Slide 43 text

‹#› @tpryan Demo: App Engine

Slide 44

Slide 44 text

‹#› @tpryan 05 Conclusion What did I learn

Slide 45

Slide 45 text

@briandorsey Wait, you’re giving up realtime without a fight?

Slide 46

Slide 46 text

‹#› @tpryan 05 Fix 3 Brining back the realtime

Slide 47

Slide 47 text

‹#› @tpryan Current / //Write to Memcache //Write to Cloud Storage //Receive realtime
 //updates /visualizer http://load-demo.appspot.com //Display Updates //Store request details

Slide 48

Slide 48 text

‹#› @tpryan Current / //Write to Memcache //Write to Cloud Storage //Receive realtime
 //updates /visualizer http://load-demo.appspot.com //Display Updates //Store request details //Read memcache //Write to Firebase /realtime.php

Slide 49

Slide 49 text

‹#› @tpryan Pseudo Code // Get list of instances // Get list of number of requests for each instance // Send data for each request to firebase Bad Idea

Slide 50

Slide 50 text

‹#› @tpryan How Firebase works • appengine • instance1 • request1 • request2 • request3 • request4 • request5 • request6 • instance2

Slide 51

Slide 51 text

‹#› @tpryan How Firebase works { "appengine": { "instance1": { "request1": "", "request2": "", "request3": "", "request4": "", "request5": "", "request6": "" }, "instance2": {} } }

Slide 52

Slide 52 text

‹#› @tpryan How Firebase works { "appengine": { "instance1": { "request1": "", "request2": "", "request3": "", "request4": "", "request5": "", "request6": "" }, "instance2": {} } } /appengine/instance1/request2.json GET POST PATCH

Slide 53

Slide 53 text

‹#› @tpryan How Firebase works { "appengine": { "instance1": { "request1": "", "request2": "", "request3": "", "request4": "", "request5": "", "request6": "" }, "instance2": {} } } /appengine/instance1.json GET POST PATCH

Slide 54

Slide 54 text

‹#› @tpryan How Firebase works { "appengine": { "instance1": { "request1": "", "request2": "", "request3": "", "request4": "", "request5": "", "request6": "" }, "instance2": {} } } /appengine.json GET POST PATCH

Slide 55

Slide 55 text

‹#› @tpryan

Slide 56

Slide 56 text

‹#› @tpryan Batch Memcache calls getMulti()

Slide 57

Slide 57 text

‹#› @tpryan Let’s send too much stuff • appengine • 00c61b117c1d99526c06775beef85526a00dbdd295ffef80dbe702a836b0 • request1 • 00c61b117c1d99526c06775beef85526a00dbdd295ffef80dbe702a836b0 • request2 • 00c61b117c1d99526c06775beef85526a00dbdd295ffef80dbe702a836b0 • request3 • 00c61b117c1d99526c06775beef85526a00dbdd295ffef80dbe702a836b0 • request4 • 00c61b117c1d99526c06775beef85526a00dbdd295ffef80dbe702a836b0 • request5 • 00c61b117c1d99526c06775beef85526a00dbdd295ffef80dbe702a836b0 • request6 • 00c61b117c1d99526c06775beef85526a00dbdd295ffef80dbe702a836b0 • 00c61b117cf44784cd0c142018b2ada185ce6d3472a963342388ef123a30

Slide 58

Slide 58 text

‹#› @tpryan Let’s send too much stuff • appengine • 00c61b117c1d99526c06775beef85526a00dbdd295ffef80dbe702a836b0 • 6 • 00c61b117cf44784cd0c142018b2ada185ce6d3472a963342388ef123a30 • 1

Slide 59

Slide 59 text

‹#› @tpryan

Slide 60

Slide 60 text

‹#› @tpryan

Slide 61

Slide 61 text

‹#› @tpryan

Slide 62

Slide 62 text

‹#› @tpryan

Slide 63

Slide 63 text

‹#› @tpryan

Slide 64

Slide 64 text

‹#› @tpryan Source Code $task1 = new PushTask('/realtime.php'); $task1->add("realtimesynch");

Slide 65

Slide 65 text

‹#› @tpryan Demo: App Engine

Slide 66

Slide 66 text

‹#› @tpryan 06 Conclusion What did I learn

Slide 67

Slide 67 text

@briandorsey Safety Quotas are not just something to be gotten around

Slide 68

Slide 68 text

@briandorsey Bundle up data calls when you can

Slide 69

Slide 69 text

@briandorsey Be prepared to make compromises

Slide 70

Slide 70 text

@briandorsey Time for a language change?

Slide 71

Slide 71 text

‹#› @tpryan Thank You terrenceryan.com @tpryan This preso: http://bit.ly/tpryan-30m