카카오 Druid 반정형 데이터 고군분투기 if(kakao)2022 Copyright 2022. Kakao Corp. All rights reserved. Redistribution or public display is not permitted without written permission from Kakao.
Simple Schema Column name Data type __time Timestamp custom_data Varchar Column name Data type __time Timestamp custom_data.srcIP Varchar custom_data.dstIP Varchar custom_data.packets Varchar custom_data.id Long custom_data.order_id Long custom_data.item Varchar
우리가 필요한 기능 JsonPath What we need Single Value Extraction O O Nested JSON Object Handling O O Polymorphic Type Handling O X Operators O X Functions O X Filter Operators O X Predicates O X
Field List {"event":{”id”:1104, “status":"N"} .id .status Type List Long String .id Local Dictionary 0 1 2 … Bitmap Index Value Column 0 1 2 … 0 0 1 … .status Local Dictionary NULL “E” “N” … Bitmap Index Value Column NULL “E” “N” … 0 0 1 …
Field List .id .status Type List Long String .id Local Dictionary 0 1 2 … Bitmap Index Value Column 0 1 2 … 0 0 1 … .status Local Dictionary NULL “E” “N” … Bitmap Index Value Column NULL “E” “N” … 0 0 1 … {"event":{”id”:1104, “status":"N"}
성능 비교 0 1200 2400 3600 4800 6000 JsonPath FastJson N estedC olum n JsonPath FastJson N estedC olum n key index 장비 스펙 core 10, mem 32GB, SSD 데이터 1,000,000 rows (ms)