Integrated Cache Training – Netscaler Webinar Mark Hillick

2 © 2007 Citrix Systems, Inc. — All rights reserved •  Goals of Caching q Why & What •  Cache Policies •  How long to cache •  Memory for Cache •  IC configuration example •  Cache Statistics •  Troubleshooting IC Agenda of Cash, sorry Cache

3 © 2007 Citrix Systems, Inc. — All rights reserved Please be aware that this a technical presentation with lots of detail and it is intended that it can be used as a reference guide at a later date (e.g. when you're on-site). Sadly, it`s not like my usual Zen presentation with lots of funny pictures and jokes, sorry L Presentation Goal

4 © 2007 Citrix Systems, Inc. — All rights reserved •  Caching would be useless if it did not significantly improve performance. •  Goal => to eliminate the need to send requests in many cases. q Reduces the number of network round-trips required for many operations - we use an "expiration" mechanism for this purpose •  Goal => remove the need to send full responses in many other cases. q  The latter reduces network bandwidth requirements - we use a "validation" mechanism for this purpose Caching Goals

5 © 2007 Citrix Systems, Inc. — All rights reserved •  Local cache in your browser •  Cache in the office (Forward Proxy or Transparent) •  Cache at the ISP (Transparent) •  Cache at the origin (Reverse Proxy or Transparent) •  Cache integrated inside a traffic manager – much better J Caching is Used Everywhere

6 © 2007 Citrix Systems, Inc. — All rights reserved Caching Benefits. 1.  Saves Bandwidth 2.  Reduces traffic to back-end server & so saves processing resources 3.  Protects back-end servers from flash crowds 4.  Improves the response time & Increases site capacity P.S. Even more benefits when integrated within a LB J

7 © 2007 Citrix Systems, Inc. — All rights reserved IC in Packet-Processing TCP Processing SSL Decrypt HTTP Parsing AAA Processing Cache Redirection Content Switching Application Firewall Responder Action? Cache Hit? HTTP Cache Req Policies Rewrite Policy Eval Load Balancing Content Filtering

8 © 2007 Citrix Systems, Inc. — All rights reserved Proxy GET /foo/index.html HTTP/1.1 Host: www.example.com HTTP/1.1 200 OK Last-Modified: Thu, ... Content-Length: 3688 Content-Type: text/html HTTP/1.1 200 OK Last-Modified: Thu, ... Content-Length: 3688 Content-Type: text/html Non-Caching Proxy GET /foo/index.html HTTP/1.1 Host: www.example.com

9 © 2007 Citrix Systems, Inc. — All rights reserved GET /foo/index.html HTTP/1.1 Host: www.example.com HTTP/1.1 200 OK Last-Modified: Thu, ... Content-Length: 3688 Content-Type: text/html GET /foo/index.html HTTP/1.1 Host: www.example.com Proxy HTTP/1.1 200 OK Last-Modified: Thu, ... Content-Length: 3688 Content-Type: text/html Proxy  Cache (Saves  copy) Caching Proxy: Miss

10 © 2007 Citrix Systems, Inc. — All rights reserved GET /foo/index.html HTTP/1.1 Host: www.example.com Proxy HTTP/1.1 200 OK Last-Modified: Thu, ... Content-Length: 3688 Content-Type: text/html Proxy  Cache (Fresh  copy!) Caching Proxy: Hit

11 © 2007 Citrix Systems, Inc. — All rights reserved HTTP Response after a hit •  wget -S --header="If-None-Match:23834-b1-4951a45768b8d" -U Mozilla http://10.90.196.68/index.html •  HTTP/1.1 200 OK ……… •  Via: NS-CACHE-9.2: 65 •  ETag: "23834-b1-4951a45768b8d" •  Server: Apache/2.2.14 (Ubuntu) ......... OR ......... •  ETag: "23834-b1-4951a45768b8d" •  2011-04-26 18:22:56 ERROR 304: Not Modified. ………

12 © 2007 Citrix Systems, Inc. — All rights reserved HTTP Response after a miss •  $ wget -S --header="If-None-Match:23834-b1-4951a45768b8d" -U Mozilla http://10.90.196.68/index.html ………. •  HTTP/1.1 200 OK •  Date: Wed, 27 Apr 2011 09:30:59 GMT •  Server: Apache/2.2.14 (Ubuntu) •  Last-Modified: Mon, 15 Nov 2010 16:52:53 GMT ……….. •  Saving to: `index.html.41

13 © 2007 Citrix Systems, Inc. — All rights reserved What Not to Cache •  Cache-Control: no-store =>applies to entire message (req/resp) •  Cache-Control: no-cache =>server cannot respond with cached response •  Cache-Control: Private => not by shared cache q More info see - http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html •  POST request. •  Don`t cache if set-cookie exists •  Don`t cache all 5xx response codes, example 503 (Gateway time out) •  By default, NS ignores cache-control headers in requests -> CG Settings

14 © 2007 Citrix Systems, Inc. — All rights reserved •  Describes how long the associated representation is fresh for. q No need to send a request to server if object has not expired q Saves time •  Expires: Fri, 30 Oct 1998 14:19:41 GMT •  Cache-Control: max-age=3600 Freshness

15 © 2007 Citrix Systems, Inc. — All rights reserved •  Validation – once expired, validator can be used to check with the server if the object is still valid. q  Send conditional request and get a short “304 Not Modified” response q  Saves bandwidth •  If-Modified-Since •  If-None-Match Validation

16 © 2007 Citrix Systems, Inc. — All rights reserved GET /foo/index.html HTTP/1.1 Host: www.example.com HTTP/1.1 304 Not Modified Date: Fri, ... GET /foo/index.html HTTP/1.1 Host: www.example.com If-Modified-Since: Thu, ... Proxy HTTP/1.1 200 OK Last-Modified: Thu, ... Content-Length: 3688 Content-Type: text/html Proxy  Cache (Saves  copy) Caching Proxy: Validation

17 © 2007 Citrix Systems, Inc. — All rights reserved Cacheability Policies •  Specifies if response is cacheable. q Request or Response-based policy •  CG is DEFAULT by default J but can be changed. •  Action types q CACHE: Transaction is cacheable. Store the object q NOCACHE: Transaction is non-cacheable. q MAY_NOCACHE o  For request based policies only o  The response may be cacheable. The default if no response cacheability policy matches is to not cache the response. q MAY_CACHE o  For request based policies only o  The response may be cacheable. The default if no response cacheability policy matches is to cache the response. q INVAL o  Mark the object as invalid

18 © 2007 Citrix Systems, Inc. — All rights reserved Cache Decision Time •  Cache, Nocache, Inval q Cache decision @ request time & unless something that prevents decision like CL mistmatch, no policy eval @ response time •  MAY_CACHE, MAY_NOCACHE q Cache decision @ response time

19 © 2007 Citrix Systems, Inc. — All rights reserved Invalidate versus Expire •  Invalidate q Invalidate will invalidate the object/"all objects in contentgroup". q But the way it is done is it is optimized such a way that we don't look at each object and remove it but when you run expire command or when new request comes for the invalid object, it is then removed. q We serve a full response (200) to client regardless of cache hit. •  Expire q Only mark the object as expired q Request comes in & instead of getting full response (i.e what we do for invalid object) we will send if-modified-since/if-none-match conditional request to back-end server. i.e can receive a 304.

20 © 2007 Citrix Systems, Inc. — All rights reserved Content Group •  Every cached object => member of CG q No CG, object => Default q minhit – no. of hits before store •  Properties of objects can be controlled per contentgroup •  Dynamic versus Static - parametised •  Configurations per content-group are q Expiry settings q Portions of object considered for caching as well as Invalidation [Parameterized Caching] q Pre-fetch objects before it gets expired (event-driven) q Inserting headers (e.g. Via header) q Memory related configuration for the objects and for this whole groups

21 © 2007 Citrix Systems, Inc. — All rights reserved Flash Cache (FC) •  FC q Improves performance –> 1 req to back-end server q Concurrent requests queued on NS & single response served q set/add cache contentgroup -flashCache YES q POWERFUL q Cannot enable with PET (ensures IC polls origin before serving)

22 © 2007 Citrix Systems, Inc. — All rights reserved Some Other CG settings •  Expiry settings (blank by default) q Relexpiry – relative -> overrides server response settings q heurexpiryParam - % q Expire after complete response received •  If no expiry settings & none in response => cache miss but can do validation (non-fc) •  Cache controls q No cache depending on size or hits q Mem limit usage per group •  Works with FC J but probably not wise on significant groupsExpire after complete response received

23 © 2007 Citrix Systems, Inc. — All rights reserved Cache Policies •  Built-In (begin with “__” & “ctx_”) q Greatly enhanced in 9.3 – should be sufficient for most •  Bound globally or per vServer q Easy-To-Administer & View within Global Policy Manager •  PI-based expression engine like other NS features

24 © 2007 Citrix Systems, Inc. — All rights reserved Cache •  Built-In (begin with “__” ) q Greatly enhanced in 9.3 – should be sufficient for most •  Bound globally or per vServer q Easy-To-Administer & View within Global Policy Manager •  PI-based expression engine like other NS features

25 © 2007 Citrix Systems, Inc. — All rights reserved Verification •  Three Verification Options (again sh cache pa) q Hostname (Sufficient for Most Companies) q Hostname & IP (Default) q DNS (Most Secure)

26 © 2007 Citrix Systems, Inc. — All rights reserved Flushing the Cache •  Flush Cache q Remove stale content manually q Troubleshooting (preferred flush, disable, enable) q CLI – either CG or Object o  flush cache contentGroup content_group_html o  flush cache contentGroup ALL o  flush cache object –locator <0x….> q GUI – IC > Cache Objects •  Automatic CG flush once CG configuration is modified

27 © 2007 Citrix Systems, Inc. — All rights reserved Cache Memory Management (1) •  BSD & PPE First •  7500 with 8gb RAM & 3 PE q 2gb for kernel, 3gb for PE & max 3gb for IC q Each PE has minimum of 1gb •  17500 – 48gb mem & 11 PE q 2gb for kernel, 23gb to PE & IC can take up to 23gb q Each PE cannot exceed 3.2gb address space •  IC takes up what’s left (!>0.5)

28 © 2007 Citrix Systems, Inc. — All rights reserved Cache Memory Management (2) •  > set cache pa –memLimit 254 •  > sh cache pa | grep -i mem Memory usage limit: 10000 MBytes Memory usage limit (active value): 10000 Mbytes => reboot Maximum value for Memory usage limit: 15662 MBytes •  > sh cache contentgroup •  Memory Allocation Issues q cli> stat cache –detail | grep alloc Memory allocation failures 5741 => cache misses

29 © 2007 Citrix Systems, Inc. — All rights reserved Cache Selectors •  Best practice to configure selectors (preferred over parameters) •  Named filter – locates particular objects in CG, not exact match •  Associate selector based on the request, not with responses •  Sample selectors •  http.req.url; http.req.method; client.ip.dst etc •  Use cases q One file only cached once regardless of hostname q Two copies of same file – compressed & uncompressed q Same file, same hostname but different VIPs & CGs

30 © 2007 Citrix Systems, Inc. — All rights reserved “Not Cache” Best Practice •  Not Cache policies first q Once IC is turned on, it will begin to cache as per the default policies. o  This may cause confidential content to be cached by accidentThen cache known, good content q Start with images – jpegs, gifs q Check statistics for o  Memory allocation errors o  Hits o  Misses o  Expiries

31 © 2007 Citrix Systems, Inc. — All rights reserved Cache HTTP Callout •  Callouts to external servers q Cache saves multiple requests •  add cache policy pol_callout -rule "CLIENT.IP.SRC.EQ (127.128.129.130 )" -action CACHE –storeInGroup •  bind cache global pol_callout –priority 100 –gotoPriorityExpression END –type REQ_DEFAULT •  sh cache object –locator <0x…> HTTP callout cell: YES HTTP callout name: callout1 HTTP callout type: TEXT HTTP callout response: NO

32 © 2007 Citrix Systems, Inc. — All rights reserved Soft-Cache Expiry •  When a back-end server does not respond quickly to requests, an upstream NetScaler can serve responses from the cache. q Liberal interpretation of RFC q Protects back-end server further (e.g. lash-type event) q Serves stale data for a minimal amount of time •  Applies to 9.2.52.3 & above

33 © 2007 Citrix Systems, Inc. — All rights reserved Configuration through Wizard (1) •  Easy-to-use J

34 © 2007 Citrix Systems, Inc. — All rights reserved Configuration through Wizard (2)

35 © 2007 Citrix Systems, Inc. — All rights reserved Configuration through Wizard (3)

36 © 2007 Citrix Systems, Inc. — All rights reserved Example Configuration •  enable feature IC •  set cache parameter –memlimit 512 •  add cache contentgroup cg_image –relExpiry 600 –memLimit 100 – maxResSize 512 •  add cache policy cp1 –rule http.req.url.contains(“jpeg”) –action cache –storeingroup cg_image •  bind cache global cp1 –priority 1 –type REQ_DEFAULT cli> sh cache object 0x000000078b4200000004 cg_image //10.102.30.16:80/abc.jpeg

37 © 2007 Citrix Systems, Inc. — All rights reserved Statistics via the CLI (1) •  cli> sh cache object 0x000000078b4200000004 cg_image //10.102.30.16:80/abc.jpeg •  cli> sh cache object –locator 0x000000078b4200000004 q Address in Memory •  nscachemgr –a/-g q Shell Equivalent •  Use grep – POWERFUL for parsing output for hits, misses etc.

38 © 2007 Citrix Systems, Inc. — All rights reserved Statistics via the CLI (2)

39 © 2007 Citrix Systems, Inc. — All rights reserved Statistics via the GUI (1)

40 © 2007 Citrix Systems, Inc. — All rights reserved Statistics via the GUI (2)

41 © 2007 Citrix Systems, Inc. — All rights reserved Statistics via the GUI (3)

42 © 2007 Citrix Systems, Inc. — All rights reserved Troubleshooting Tools1 •  Browser -> inconsistencies & can be very frustrating L q FF (HTTP Fox & Tamper Data Extensions but FF4 issues) q IE (7, 8, 9) -> Fiddler q Opera & Chrome -> as standard, no extensions •  Tracing & PCAP -> q Nstrace & Wireshark as usual J

43 © 2007 Citrix Systems, Inc. — All rights reserved Troubleshooting Tools2 •  Paros (as a proxy in the middle) -> I now use Zap (Owasp project fork of Paros) •  CLI -> no user-cache to worry about J q wget •  wget –S –header=“If-None-Match:etag_value” http://test.com/file.jpg q curl o curl –header “If-None-Match: etag_value” test.com/index.html

44 © 2007 Citrix Systems, Inc. — All rights reserved Troubleshooting •  Cache turned on but no cache objects q sh cache pa != Memory usage limit: 0 bytes q Zero memory = cache misses o  Warning msg in 9.2 when bsh cache object` q Check expiry settings? NS? Server? •  Blank entries in nscachemgr output? q //10.90.196.76/ q //10.90.196.76/citrix_logo.jpg q Default caching properties of lGET / HTTP/1.1z

45 © 2007 Citrix Systems, Inc. — All rights reserved CTX Articles (just a snapshot) •  http://support.citrix.com/article/CTX124553 - how to IC •  http://support.citrix.com/article/CTX123753 - compressed content to unadvertised clients •  http://support.citrix.com/article/CTX126557 - caching uncompressed & compressed content of same file •  http://support.citrix.com/article/CTX128801 - same hostname/object but different vip twice •  http://support.citrix.com/article/CTX129118 - has my HTTP callout been cached •  http://support.citrix.com/article/CTX124718 - cache callout response •  http://support.citrix.com/article/CTX129734 - Soft Cache Expiry

46 © 2007 Citrix Systems, Inc. — All rights reserved Extra Reading •  You can use perl, curl, wget, browser add-ons but I like wget – q http://www.cyberciti.biz/tips/linux-wget-your-ultimate- command-line-downloader.html q http://www.computerhope.com/unix/wget.htm q Google “wget site:www.linuxquestions.org” q Easy to write a basic shell script around it & doesn’t need to be too complicated •  RFC 2616 (HTTP) VERY IMPORTANT q Section 13 (Caching) q Section 14 (Headers)

47 © 2007 Citrix Systems, Inc. — All rights reserved So with IC you can, take over….

48 © 2007 Citrix Systems, Inc. — All rights reserved Sláinte

49 © 2007 Citrix Systems, Inc. — All rights reserved