대용량 지표 관리를 위한 TSDB 개발기

#TSCoke

무중단 확장이 가능한 prometheus 호환 Time series DBMS인 TSCoke에 대한 소개와, 성능 최적화 과정에 대해 소개합니다.

발표자 : yanoo.kim
카카오 분산기술셀에서 분산 스토리지 플랫폼을 개발하는 야누 입니다.

Transcript

1. ؀ਊ۝ ૑಴ ҙܻܳ ਤೠ TSDB ѐߊӝ ӣো਋ yanoo.kim ஠஠য় Copyright 2022. Kakao Corp. All rights reserved. Redistribution or public display is not permitted without written permission from Kakao. if(kakao)2022
2. ѐߊزӝ TSCoke ࣗѐ ࢿמ ୭੸ച അട ߂ ೱറ ҅ദ
3. ѐߊزӝ
4. ѐߊزӝ ؀ӏݽ ௿۞झఠܳ ҙܻೡٸ
 о੢ ݢ੷ ࠁח ചݶ?
5. 14:0015:0016:0017:0018:0019:0020:0021:00 ѐߊزӝ ੉࢚૚റఐ૑ 2ਘ 3ਘ 4ਘ 5ਘ 6ਘ 7ਘ 8ਘ оਊҕр ࢎਊ۝ 0 250 500 11ੌ 12ੌ 13ੌ 14ੌ 15ੌ 16ੌ 17ੌ ҃Ҋ য়ܨ ҳݒ҅ദࣻ݀ ࢲ࠺झಿ૕ഛੋ
6. טযաח ࢲߡ, טযաח ૑಴ ૑಴੷੢ࣗ ૑಴੷੢ࣗ ૑಴੷੢ࣗ ૑಴੷੢ࣗ ૑಴੷੢ࣗ ૑಴੷੢ࣗ
7. “੉Ҕ ੷Ҕ ৈ۞ ૑಴ٜਸ ೞա੄ ࠭۽ ೤ச ࣻח হਸө?”
8. TSCoke ࣗѐ
9. TSCoke ࣗѐ TSDB + REDI COKE
10. TSDB द҅ৌ ؘ੉ఠ ୊ܻী ୭੸ചػ DBMS TSCoke ࣗѐ
11. Redicoke Redis ೐۽ష௒ਸ ࢎਊೞח
 NAND ӝ߈ ؀ਊ۝ ࠙࢑ Key-Value झషܻ૑ TSCoke ࣗѐ
12. TSCoke ࣗѐ QD QD QD QD TSD TSD TSD TSD Redicoke REDO LOG SERIES META SAMPLE TSD ADDR INFO QD ADDR INFO ACCOUNT MAP SHARD MAP REPLICATION MGR ADMIN WATCHER TSCoke
13. WAL Replication & ޖ઺ױ ࣻಣഛ੢
14. Grafana Datasource োز Promql Compliance Test Prometheus ഐജࢿ
15. ࢿמ ୭੸ച
16. SampleChunk ୭੸ച Memory Local Storage Redicoke A B C 6ਘ A:0 B:3 C:20 A ঑୷ A ױӝ ؘ੉ఠ(~30࠙) ઺ӝ ؘ੉ఠ(~3दр) ੢ӝ ؘ੉ఠ(3दр~)
17. SampleChunk ୭੸ച Memory Local Storage Redicoke A B C 6ਘ A:0 B:3 C:20 A ঑୷ A ױӝ ؘ੉ఠ(~30࠙) ઺ӝ ؘ੉ఠ(~3दр) ੢ӝ ؘ੉ఠ(3दр~) ৘࢚ࠁ׮ ੣਷ LocalStorage I/O۽ ੹୓ WriteThroughput੉ ઁডؽ
18. SampleChunk ୭੸ച - Local Stroage ো࢑ਸ ਤ೧ BadgerDBܳ ࢎਊ - LSMT(Log-Structured Merge Tree) ഋక - LSMTח য়ېػ ؘ੉ఠ੄ ઁѢ, Range Scanҗ੿ীࢲ੄ ബਯࢿ ૐ؀ܳ ਤ೧ Compactionҗ੿ਸ ࣻ೯ೣ - Compaction਷ CPU, IO ੗ਗਸ ௼ѱ ࣗݽೣ SSTable Mem table SSTable SSTable SSTable SSTable SSTable SSTable L1 L2 L3 Compaction
19. SampleChunk ୭੸ച - Local Storageח 30࠙~3दр ࢎ੉੄ ઺ӝ ؘ੉ఠ ੷੢ࣗ. ٮۄࢲ য়ېػ ؘ੉ఠо হ਺ 
 -> Compaction ױࣽച оמ - Read, FullScan݅ ೙ਃೡࡺ, RangeScan਷ ೙ਃ হ਺.
 -> ੿۳ җ੿ ೙ਃ হ਺
 -> SSTable Merge ೙ਃ হ਺ SSTable Mem table SSTable SSTable SSTable SSTable SSTable SSTable L1 L2 L3 Compaction
20. SampleChunk ୭੸ച - LSS(Log-structured Segment Store) ਬബೠ ؘ੉ఠ ݅ܐػ ؘ੉ఠ Append Tail Block Truncate Block Block Block Head Block
21. SampleChunk ୭੸ച 0 900 1800 Badger LSS IOPS ݫݽܻ ࢎਊ۝ CPU: Intel(R) Xeon(R) CPU E5-2620 v4 @ 2.10GHz 
 SSD: Samsung - PM9A3 U.2 ֫ਸ ࣻ۾ જ਺ ծਸ ࣻ۾ જ਺ 0 250000 500000 Badger LSS
22. SeriesIndex ୭੸ച 15.13GB 9.23GB 18.67GB SeriesIndex Others SampleChunk ೠ ֢٘੄ ݫݽܻ ࣗݽ۝
23. SeriesIndex ୭੸ച metric{method=“used”,instance=“192.168.2.101”} metric{method=“used”,instance=“192.168.2.117”} metric{method=“used”,instance=“192.168.2.163”} sum by (method)(rate(metric{method=“used”}[5m]))
24. SeriesIndex ୭੸ച <SeriesID:1000,1001,1002> metric,method,used = <SeriesID:1000>, <SeriesID:1001>, <SeriesID:1002> metric,method,free = <SeriesID:1003>, <SeriesID:1004>, <SeriesID:1005> metric,instance,192.168.2.101 = <SeriesID:1000> metric,instance,192.168.2.117 = <SeriesID:1001> metric,instance,192.168.2.163 = <SeriesID:1002> metric,instance,192.168.2.185 = <SeriesID:1003> metric,instance,192.168.2.214 = <SeriesID:1004> metric,instance,192.168.3.28 = <SeriesID:1005> Metric, LabelName, LabelValue Index sum by (method)(rate(metric{method=“used”}[5m]))
25. SeriesIndex ୭੸ച 00 01 10 11 00 01 10 11 00 01 10 11 Radix Search Radix Tree 00 11 0000 0001 0010 1100 1110 1111 Binary Search BTree
26. SeriesIndex ୭੸ച Dictionary Coding John,Male,Seoul Mary,Female,London Jane,Female,Seoul Peter,Male,Tokyo Sam,Male,London Emily,Female,Tokyo 1,2,3 4,5,6 7,5,3 8,2,9 10,2,6 11,5,9 John=1 Male=2 Seoul=3 Mary=4 Female=5 London=6 Jane=7 Peter=8 Tokyo=9 Sam=10 Emily=11
27. SeriesIndex ୭੸ച metric,method,used = 1000,1001,1002 metric,method,free = 1003,1004,1005 metric,instance,192.168.2.101 = 1000 metric,instance,192.168.2.117 = 1001 metric,instance,192.168.2.163 = 1002 metric,instance,192.168.2.185 = 1003 metric,instance,192.168.2.214 = 1004 metric,instance,192.168.3.28 = 1005 Metric, LabelName, LabelValue Index Table metric=1 method=2 used=3 free=4 instance=5 192.168.2.101=6 192.168.2.117=7 192.168.2.163=8 192.168.2.185=9 192.168.2.214=10 192.168.3.28=11 1,2,3 = 1000,1001,1002 1,2,4 = 1003,1004,1005 1,5,6 = 1000 1,5,7 = 1001 1,5,8 = 1002 1,5,9 = 1003 1,5,10 = 1004 1,5,11 = 1005 Metric, LabelName, LabelValue Index Dictionary Coding
28. SeriesIndex ୭੸ച A B A B A B A B A B A B A B Dictionary Coding 0 1 2 3 4 5 6 7 AAA AAB ABA ABB BAA BAB BBA BBB 0=AAA 1=AAB 2=ABA 3=ABB 4=BAA 5=BAB 6=BBA 7=BBB
29. SeriesIndex ୭੸ച 0 1 2 sync.M ap ART sync.M ap+D IC T ART+D IC T Index ઑഥ दр ծਸ ࣻ۾ જ਺ 0 2500 5000 sync.M ap ART sync.M ap+D IC T ART+D IC T ݫݽܻ ࢎਊ۝ ծਸ ࣻ۾ જ਺
30. ࠙࢑ ૕੄ TBә ੷੢۝ = TB ઑഥ۝
31. ࠙࢑ ૕੄ 1,600ѐ 4઱/15ୡ=161,280 1,600*161,280 = 258,048,000 ѐ
32. ࠙࢑ ૕੄ QD QD QD QD
33. ࠙࢑ ૕੄ ਽׹दр(sec) ծਸ ࣻ۾ જ਺ 0 15 30 1؀ 2؀ 3؀ 4؀ 5؀ 6؀ 7؀ 8؀ CPU: Intel(R) Xeon(R) CPU E5-2620 v4 @ 2.10GHz 
 SSD: Samsung - PM9A3 U.2
34. അട ߂ ೱറ ҅ദ
35. ࢲ࠺झ അട 2022/06/09 ࢎղ ࢲ࠺झ য়೑ - ࢲ࠺झ҅੿ 126ѐ - series ѐࣻ 4রѐ - DKOS(ࢎղ Kubernates)ܳ ੉ਊ೧ ࢲ࠺झ ઺ - TSCoke as a Service
36. ೱറ ҅ദ - Alarm, Recording Rule ӝמ - ৻ࠗ ࠙࢑ झషܻ૑ োز ӝמ - Down-sampling
37. хࢎ೤פ׮
38. ଵҊ ޙ೴ 1) Badger(github.com/dgraph-io/badger) 2) Leis, Viktor, Alfons Kemper, and Thomas Neumann. “The adaptive radix tree: ARTful indexing for main-memory databases.” 2013 IEEE 29th International Conference on Data Engineering (ICDE). IEEE, 2013.