HeapOff memory WTF

HeapOff caching with Apache DirectMemory

Transcript

1. Heap Off Memory WTF ?? Reducing Heap memory stress
2. Abstract * Java memory fundamental * heap off memory principles * heap-off cache with Apache DirectMemory
3. /me Olivier Lamy * Open Source Architect @Talend Apache Team * Apache Member : Maven, Archiva, Tomcat, DirectMemory, Build Infra, etc.. * Jenkins * Add bugs in various OpenSource projects * twitter.com/olamy olamy.blogspot.com
4. Java Memory Fundamental
5. Java Memory Fundamental The cool part * Automatic memory allocation (do you remember when you used malloc) * Garbage collector (GC) (no more free)
6. Java Memory Fundamental The bad part * stop-the-world mode * proportionnal to memory's size * unpredictable unresponsive application : complicated with tight SLA
7. Heap Off Memory Not in the Heap process Limited GC stop « Unlimited storage »
8. Heap Off Memory How to use that ? Native libraries with JNA https://github.com/twall/jna ByteBuffer @since 1.4 with nio ByteBuffer.allocate( size ) (not off heap) ByteBuffer.allocateDirect( size ) You can use sun.misc.Unsafe but can cause issue and not portable
9. Heap Off Memory How to use that ? You can manipulate only Buffer (ByteBuffer byte[] etc..) ! So you must serialize/deserialize your datas Maximum size with -XX:MaxDirectMemorySize=
10. Our use case : Cache RAM : 10-60 ns Network : 10000-30000 ns SSD Disk : 70000-120000 ns Disk : 3000000-10000000 ns Cache must use memory !
11. On-Heap Cache Objects stored by reference (no de/serialisation) GC storm effect when refreshing/removing objects!
12. Off-Heap Cache de/serialisation overhead (hopefully some very performant libraries exist for that) Cache objects payload no more affecting GC
13. Solutions : * Terracotta BigMemory (off-heap storage on top of ehcache) * Infinispan (by Jboss) * Huge collections * Apache DirectMemory
14. Apache Direct Memory Goals : Apache Direct Memory is a multi layered cache implementation featuring off-heap memory storage to enable caching of java objects without degrading jvm performance.
15. Apache Direct Memory • Joined Apache Incubator end 2011 • 12 developpers ATM • Under development : – Memory allocation service just rewrite – APIs are subject to be changed and bugs to be found !
16. Design & principles • ByteBuffer.allocateDirect is the foundation of the cache • ByteBuffers are allocated in big chunk and splitted for internal use
17. Design & principles Build on layers – CachingService : serialize object (pluggable) – MemoryManagerService: Compute ByteBuffer access – ByteBufferAllocatorService: Eventually deals with ByteBuffer
18. ByteBuffers Allocation Merging Strategy – No memory wasted – Suffers from fragmentation – Need synchronization at de/allocation Fixed Size buffers allocation – Memory wasted if size not correctly configured – No fragmentation
19. Use case (1) Multi layers cache • Most used objects are cached on heap, the rest off-heap (maybe overflow to disk) • Sounds like ehcache with BigMemory • Hard coded class to use : net.sf.ehcache.store.offheap.OffHeapStore • So same package class name in Apache DirectMemory (https://jira.terracotta.org/jira/browse/EHC-940) • Demo
20. Use case (2) Cache Server • « à la » memcache tru http(s) • But with a REST Api and written in Java • Client API available • Demo
21. > PUT /dm/cache/bordeaux HTTP/1.1 > Content-Type:text/plain {"millesime":"2003","description":"so good so good"} < HTTP/1.1 200 OK < X-DirectMemory-SerializeSize: 58 < Content-Length: 0 > GET /dm/cache/bordeaux HTTP/1.1 > Accept:text/plain < HTTP/1.1 200 OK < Content-Type: text/plain < Content-Length: 51 {"millesime":"2003","description":"so good so good"} > DELETE /dm/cache/foo HTTP/1.1
22. Next Steps • JSR 107 ? • Benchmarks • Components Integration (Cassandra, Tomcat etc..) • Dynamic cache size modification • Management/Monitoring
23. Links • http://incubator.apache.org/directmemory/ • https://issues.apache.org/jira/browse/DIRECTMEMORY • Demo project : https://bitbucket.org/olamy/heap-off- memory-wtf • Slides : http://www.slideshare.net/olamy
24. Thanks && Questions
25. Heap Off Memory WTF ?? Reducing Heap memory stress
26. Abstract * Java memory fundamental * heap off memory principles * heap-off cache with Apache DirectMemory
27. /me Olivier Lamy * Open Source Architect @Talend Apache Team * Apache Member : Maven, Archiva, Tomcat, DirectMemory, Build Infra, etc.. * Jenkins * Add bugs in various OpenSource projects * twitter.com/olamy olamy.blogspot.com
28. Java Memory Fundamental
29. Java Memory Fundamental The cool part * Automatic memory allocation (do you remember when you used malloc) * Garbage collector (GC) (no more free)
30. Java Memory Fundamental The bad part * stop-the-world mode * proportionnal to memory's size * unpredictable unresponsive application : complicated with tight SLA
31. Heap Off Memory Not in the Heap process Limited GC stop « Unlimited storage »
32. Heap Off Memory How to use that ? Native libraries with JNA https://github.com/twall/jna ByteBuffer @since 1.4 with nio ByteBuffer.allocate( size ) (not off heap) ByteBuffer.allocateDirect( size ) You can use sun.misc.Unsafe but can cause issue and not portable
33. Heap Off Memory How to use that ? You can manipulate only Buffer (ByteBuffer byte[] etc..) ! So you must serialize/deserialize your datas Maximum size with -XX:MaxDirectMemorySize=
34. Our use case : Cache RAM : 10-60 ns Network : 10000-30000 ns SSD Disk : 70000-120000 ns Disk : 3000000-10000000 ns Cache must use memory !
35. On-Heap Cache Objects stored by reference (no de/serialisation) GC storm effect when refreshing/removing objects!
36. Off-Heap Cache de/serialisation overhead (hopefully some very performant libraries exist for that) Cache objects payload no more affecting GC
37. Solutions : * Terracotta BigMemory (off-heap storage on top of ehcache) * Infinispan (by Jboss) * Huge collections * Apache DirectMemory
38. Apache Direct Memory Goals : Apache Direct Memory is a multi layered cache implementation featuring off-heap memory storage to enable caching of java objects without degrading jvm performance.
39. Apache Direct Memory • Joined Apache Incubator end 2011 • 12 developpers ATM • Under development : – Memory allocation service just rewrite – APIs are subject to be changed and bugs to be found !
40. Design & principles • ByteBuffer.allocateDirect is the foundation of the cache • ByteBuffers are allocated in big chunk and splitted for internal use
41. Design & principles Build on layers – CachingService : serialize object (pluggable) – MemoryManagerService: Compute ByteBuffer access – ByteBufferAllocatorService: Eventually deals with ByteBuffer
42. ByteBuffers Allocation Merging Strategy – No memory wasted – Suffers from fragmentation – Need synchronization at de/allocation Fixed Size buffers allocation – Memory wasted if size not correctly configured – No fragmentation
43. Use case (1) Multi layers cache • Most used objects are cached on heap, the rest off-heap (maybe overflow to disk) • Sounds like ehcache with BigMemory • Hard coded class to use : net.sf.ehcache.store.offheap.OffHeapStore • So same package class name in Apache DirectMemory (https://jira.terracotta.org/jira/browse/EHC-940) • Demo
44. Use case (2) Cache Server • « à la » memcache tru http(s) • But with a REST Api and written in Java • Client API available • Demo
45. > PUT /dm/cache/bordeaux HTTP/1.1 > Content-Type:text/plain {"millesime":"2003","description":"so good so good"} < HTTP/1.1 200 OK < X-DirectMemory-SerializeSize: 58 < Content-Length: 0 > GET /dm/cache/bordeaux HTTP/1.1 > Accept:text/plain < HTTP/1.1 200 OK < Content-Type: text/plain < Content-Length: 51 {"millesime":"2003","description":"so good so good"} > DELETE /dm/cache/foo HTTP/1.1
46. Next Steps • JSR 107 ? • Benchmarks • Components Integration (Cassandra, Tomcat etc..) • Dynamic cache size modification • Management/Monitoring
47. Links • http://incubator.apache.org/directmemory/ • https://issues.apache.org/jira/browse/DIRECTMEMORY • Demo project : https://bitbucket.org/olamy/heap-off- memory-wtf • Slides : http://www.slideshare.net/olamy
48. Thanks && Questions