Cache me if you can: memcached, caching patterns and best practices by Guillaume Ardaud

Slide 1

Slide 1 text

CACHE ME IF YOU CAN Guillaume Ardaud  @gardaud | gardaud.fr  [email protected] Friday April 11th  PyCon2014  Montreal

Slide 2

Slide 2 text

MEMCACHED -What is it?  -How does it work?  -Patterns & Best Practices

Slide 3

Slide 3 text

Key/Object Store • Give it an object, identiﬁed by a key  • Ask for a key  • If it isn’t expired, you get the object

Slide 4

Slide 4 text

O(1) EVERYTHING

Slide 5

Slide 5 text

primary data:  relational database data you can lose, regenerate fast:   RAM store data you can lose, regenerate slowly:   persistent store Where does memcached ﬁt?

Slide 6

Slide 6 text

Typical memcached API

Slide 7

Slide 7 text

No content

Slide 8

Slide 8 text

No content

Slide 9

Slide 9 text

No content

Slide 10

Slide 10 text

No content

Slide 11

Slide 11 text

No content

Slide 12

Slide 12 text

Key names - ASCII based  - Not crazy long…  - … but explicit enough

Slide 13

Slide 13 text

Good naming e.g.:   json.users.

Slide 14

Slide 14 text

Bad naming e.g.:   md5(sql_query)

Slide 15

Slide 15 text

HOW DO I LIST ALL KEYS IN MEMCACHED? YOU DON’T (or all keys matching a regex, or all...)

Slide 16

Slide 16 text

it’s a cache not a database

Slide 17

Slide 17 text

No content

Slide 18

Slide 18 text

memcached docs:

Slide 19

Slide 19 text

No content

Slide 20

Slide 20 text

memcached is distributed

Slide 21

Slide 21 text

your amazing application memcached box A memcached box B memcached box C memcached client

Slide 22

Slide 22 text

your amazing application memcached box A memcached box B memcached box C memcached client set $KEY send $KEY  to box C Received!

Slide 23

Slide 23 text

your amazing application memcached box A memcached box B memcached client set $KEY send $KEY  to box A

Slide 24

Slide 24 text

~ some memcached subtleties ~

Slide 25

Slide 25 text

EXPIRATION It’s 8am, a request stores an object under the key ‘foo’, with a 2 hour expiration time. What happens at 10am?

Slide 26

Slide 26 text

But what happens when a client tries to access 'foo' at 1pm? …'foo' gets removed. NOTHING

Slide 27

Slide 27 text

Q: Sometimes fetching a key returns None even though it’s not supposed to have expired yet?

Slide 28

Slide 28 text

Memcached has a ﬁxed amount of memory." If you try to store data when memory is full, data will get evicted.

Slide 29

Slide 29 text

How does memcached determine what to evict? Object 1 Object 2 timestamp: 1378399504 timestamp: 137839983 Object 3 timestamp: 1378399939 Object 4 timestamp: 13783992932 Object 5 timestamp: 13783993829

Slide 30

Slide 30 text

Q: Sometimes fetching a key returns None even though it’s not supposed to have expired yet… … AND memcached still has free memory?

Slide 31

Slide 31 text

No content

Slide 32

Slide 32 text

page 1 page 2 page 3 page 4 page 5 page 6 page 7 ... 1 page = 1MB

Slide 33

Slide 33 text

page chunk chunk chunk chunk chunk chunk chunk chunk chunk chunk chunk chunk chunk chunk chunk chunk 1st_slab_chunk_size = 80 bytes! 2nd_slab_chunk_size = 80 * 1.25 = 100 bytes! 3rd_slab_chunk_size = 100 * 1.25 = 125 bytes! . . .! last_slab_chunk_size = 1048576 bytes Last slab has 1 chunk of 1 MB.

Slide 34

Slide 34 text

No content

Slide 35

Slide 35 text

page 1 page 2 page 3 page 4 page 5 page 6 page 7 ... class 1 class 2 class 3 class 4 class 5 free page free page

Slide 36

Slide 36 text

No content

Slide 37

Slide 37 text

If all pages of the needed class are full: get a free page and give it the needed class. If there are no more free pages:   the LRU kicks in and evicts data. Each slab class has its own LRU! " If your data needs class 3 and: - there are no free pages - all class 3 pages are full - some class 4 pages have free chunks Data in a class 3 page will still be evicted.

Slide 38

Slide 38 text

Q: Sometimes fetching a key returns None even though it’s not supposed to have expired yet… … AND memcached still has free memory?

Slide 39

Slide 39 text

some useful command line ﬂags memcached -v : verbose output (more verbose: -vv) memcached -M: doesn’t evict when out of memory, but errors " memcached -I1k : change slab page size (1k - 128m) memcached -I1m! memcached -I128m! " memcached -f1.5 : change growth factor (def. 1.25, >1) " " man memcached is your friend

Slide 40

Slide 40 text

Let’s get our hands dirty. common memcached patterns & practices in Python

Slide 41

Slide 41 text

No content

Slide 42

Slide 42 text

No content

Slide 43

Slide 43 text

No content

Slide 44

Slide 44 text

MODEL VERSIONING

Slide 45

Slide 45 text

FETCH STRAIGHT FROM CACHE

Slide 46

Slide 46 text

No content

Slide 47

Slide 47 text

THUNDERING HERD

Slide 48

Slide 48 text

No content

Slide 49

Slide 49 text

CACHING LARGE VALUES

Slide 50

Slide 50 text

2-PHASE FETCH

Slide 51

Slide 51 text

1. Break big object in smaller chunks (for example chunks of 10 books) PAGINATED CACHE 2. Store each chunk as a separate object in memcached 3. Store the list of keys To fetch the data, fetch the list of keys, iterate through it, and concatenate all fetched lists.

Slide 52

Slide 52 text

No content

Slide 53

Slide 53 text

No content

Slide 54

Slide 54 text

No content

Slide 55

Slide 55 text

Questions? THANKS!

Slide 56

Slide 56 text

Great memcached resources Memcached doc: http://code.google.com/p/memcached/wiki/NewStart?tm=6 " Especially those 2 pages: http://code.google.com/p/memcached/wiki/NewProgrammingTricks http://code.google.com/p/memcached/wiki/NewUserInternals " Various articles/posts about internals etc.: http://stackoverﬂow.com/questions/6868256/memcached-eviction-prior-to-key- expiry http://www.adayinthelifeof.nl/2011/02/06/memcache-internals/ http://work.tinou.com/2011/04/memcached-for-dummies.html http://returnfoo.com/2012/02/memcached-memory-allocation-and-optimization-2/

Slide 57

Slide 57 text

No content