Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Performance Optimization 
on Google AppEngine

Performance Optimization 
on Google AppEngine

The presentation at GoConference 2018 Spring

5bfed9aa3a9ebccb0c0f0cb65ee9e012?s=128

Seiji Takahashi

April 15, 2018
Tweet

Transcript

  1. Performance 
 Optimization 
 on Google AppEngine Go Conference 2018

    Spring Seiji Takahashi @__timakin__
  2. Who Am I • Seiji Takahashi • @__timakin__ • Gunosy

    • Application Dev: LUCRA • Serverside Engineering (Go) • iOS Client Side (Swift) • Engieering • Language: Go (Contributed), Swift • Platform: GAE, AWS
  3. "CPVUUIJTUBML • We’ll not cover intro to AppEngine. • General

    guideline to speed up API performance on GAE.
  4. "HFOEB • Warming up • Route between internal services •

    Task Queue • Delayed Job • Batching • Caching • Asynchronous requests with goroutine
  5. "HFOEB • Warming up • Route between internal services •

    Task Queue • Delayed Job • Batching • Caching • Asynchronous requests with goroutine
  6. Warming Up Building an endpoint of warmup, You can prepare

    before serving data for user’s request. For example, you retrieve the data from datastore and store it on memcache to respond faster.
  7. BQQZBNM

  8. None
  9. "HFOEB • Warming up • Route between internal services •

    Task Queue • Delayed Job • Batching • Caching • Asynchronous requests with goroutine
  10. Shortest Route 1. No custom domain If you send an

    internal request to a different service, you should not use custom domain to decrease name resolution cost.
  11. Shortest Route 2. Turn off FollowRedirects Internal API should not

    return other than 200, 400, and 500 series statues. And you should exclude redirects with setting FollowRedirects to false.
  12. Shortest Route 3. Single Project Networking between multi-projects will add

    latency, so if you purpose the best performance, merge services into one project.
  13. "HFOEB • Warming up • Route between internal services •

    Task Queue • Delayed Job • Batching • Caching • Asynchronous requests with goroutine
  14. Task Queue A heavy job should be scheduled with Task

    Queue, not user’s request. And it’ll be effective on cron job like crawling too.
  15. ex) User’s request Client GAE GAE Job Job Job Job

    Push Queue Immediate Response
  16. ex) Cron job GAE GAE Job Job Job Job Push

    Queue Bulk Insert Cron
  17. RVFVFZBNM

  18. None
  19. "HFOEB • Warming up • Route between internal services •

    Task Queue • Delayed Job • Batching • Caching • Asynchronous requests with goroutine
  20. Delayed Job delay package is also background job handling system.

    It depends on Task Queue. It simplify the asynchronous API of Task Queue, and it can receive arguments of struct.
  21. Delayed Job delay package is also background job handling system.

    It depends on Task Queue. It simplify the asynchronous API of Task Queue, and it can receive arguments of struct. default Task Queue runs tasks through HTTP request and can pass only a string query parameter.
  22. *Caution delay.Func will search an execution target with file name

    and task key, so if you change the file name, waiting tasks will drop because it can’t find the func. Additionally, it must be declared on init() or as a global variable. If it was declared inside of request handling, a receiver instance may fail because Func wasn’t initialized just after spinning-up.
  23. None
  24. "HFOEB • Warming up • Route between internal services •

    Task Queue • Delayed Job • Batching • Caching • Asynchronous requests with goroutine
  25. Batching Executing Get/Put/Delete for multi resources in the loop will

    generate RPC calls. If you can get keys for the records, reduce them with GetMulti/PutMulti/DeleteMulti. And Task Queue also supports AddMulti for bulk insertion.
  26. None
  27. None
  28. None
  29. Caching Storing/Fetching data from Memcache is better for performance. The

    cache will be shared among instances. Some third-party packages are good for using Datastore and Memcache simultaneously. • mjibson/goon • mercari/datastore
  30. Caching Almost all of interfaces of these packages is same.

    But goon wins on the current reading benchmark.
  31. None
  32. "HFOEB • Warming up • Route between internal services •

    Task Queue • Delayed Job • Batching • Caching • Asynchronous requests with goroutine
  33. Asynchronous requests with goroutine AppEngine environment forces GOMAXPROCS=1. But within

    I/O wait time, Go will schedule to other goroutine. Like Datastore/Search API call can be artificially asynchronous and it performs excellent latency.
  34. Asynchronous requests with goroutine Basically GetMulti/PutMulti is effective. But when

    you cannot know the datastore.Key before fetching. For example, goroutine is better for query filtering.
  35. None
  36. None
  37. goroutine in goroutine is sensitive technique from the aspect of

    error handling, but good for performance.
  38. 4VNNBSZ • AppEngine is tunable with routing, background job, goroutine,

    batch, cache, etc… • Async func is really attractive if your primary goal is performance. However, you must take care of error handling and initialization.
  39. 5IBOLZPV