Upgrade to Pro — share decks privately, control downloads, hide ads and more …

ZGC for Future LINE Base

ZGC for Future LINE Base


LINE Developers

September 05, 2019

More Decks by LINE Developers

Other Decks in Programming


  1. About me • Shinya Yoshida • HBase Unit, Z Part,

    Dept1, Dev1, LINE JP server Z Part Other services
  2. Agenda • Garbage Collection/or • How to choose GC •

    Evaluate HBase with ZGC • Applying ZGC to production HBase
  3. Garbage Collection/or • How to collect garbage • How to

    use heap spaces – Generational, Regioning – Skip this topic for time reason • Stop The World(STW) – World == Application – Application cannot do nothing while in STW
  4. HBase and GC • HBase = NoSQL running on JVM

    – JVM pause time(STW) by GC Worst response time ⇒ Worst – Response time Service ⇒ Worst • We use HBase for persistent store in LINE Messaging – So, Response time of HBase is important
  5. Basic GC Algorithm idea • Finding non referenced objects(garbage) –

    Counting reference – Mark • Collecting – Sweep/Compaction • Remove and defrag – Copy(Relocate) • Move live objects to another space • Do either while or after marking
  6. There are fragmentation Fragmentation is solved, but the address will

    be changed Need another space Sweep/Compaction Copy
  7. GC in Java GC Algorithms How to collect Young GC

    Sequential/Parallel Copy Old GC Sequential/Parallel Sweep&Compact Concurrent Mark & Sweep (To be removed) Sweep only (No compaction) G1GC Copy ZGC(Java11, Experimental) Copy Shenandoah(Java12, Experimental) Copy
  8. GC with running application • Yes, we can – Known

    as Concurrent GC, however • Need to stop application(STW) while GC for – Soundness(Must):Never collect living object – Completeness(Option):Collect all garbage • STW duration depends on algorithm – How long, often – Scaling factor(Heap size? # of live objects? # of threads?)
  9. STW and scaling factor of each GCs GC Algorithms Mark

    Collect Old GC Sequential Parallel CMS(To be removed) Full GC somewhen G1GC Scales by heap size, live objects, etc, but manageable STW because address can be changed ZGC Scales by # of Threads STW:<=10ms? Shenandoah STW:<=10ms? Running application(e.g. Concurrent) STW GC cycles
  10. ZGC and Shenandoah • Core idea is mostly same between

    two – Changing object address/pointer while running GC and application • Check official slides for more details – https://wiki.openjdk.java.net/display/zgc/Main
  11. Choosing GC • Know GC’s properties – Pros/Cons • Consider

    your application properties and HWs • Choose better one
  12. What’s GC’s properties • Response time(STW duration) / Throughput? •

    How many CPU cores are required for best performance • Heap size • Live object ratio for best performance • Object sizes • And so on
  13. Properties for each Java GCs ∞ 85% 100% 1GB 8GB

    512GB 8TB〜 0ms 100ms 1s 1 2 4 8 16 32+ 31GB 128GB 2TB Sequential/Parallel G1GC Shenandoah/ZGC STW Throughput Preferred # of CPU cores Preferred heap size
  14. What about our HBase? • Response time is important than

    throughput – We may scale out(Need money though) • Want large heap for caching data • Don’t use CPU so much by HBase itself – Don’t use Phoenix(SQL for HBase) – Some compaction uses CPU core fully, but # of thread is small – Mostly IO and Networks • Server specs – 40 CPU cores – 256GB RAM
  15. Our HBase properties vs GC properties STW Throughput Preferred #

    of CPU cores Preferred heap size ∞ 85% 100% 1GB 8GB 512GB 8TB〜 0ms 100ms 1s 1 2 4 8 16 32+ 31GB 128GB 2TB Sequential/Parallel G1GC Shenandoah/ZGC
  16. What we did • Construct HBase1.2.5+α on test cluster •

    Running it on Java11-ea – We used HBase built by JDK8 • Enable ZGC • Eval performance using YCSB, a benchmark tool – STW duration and slow response of HBase – Throughput – As contrast, G1GC
  17. RPC Response JDK8 + G1GC Consumer RPC Info JDK11 +

    ZGC RPC Response server WORK IN PROGRESS
  18. Conclusions • New GCs coming to Java – ZGC、Shenandoah: Still

    Experimental – Choose it understanding properties of GC, your app and HW • We like ZGC – Shows better performance with our HBase • Evaluate it more using production requests – Before apply ZGC to our production HBase