q and kdb+

TDQGNGE IURPYHFWRUSURFHVVLQJWRFROXPQRULHQWHGGDWDEDVH

7KHSUREOHP P\VWUDWHJ\XVXDOO\LVEX\VHOO'DLPOHULI FKHDSFRPSDUHGWR%0:DQG9RONVZDJHQ ZKHQ9:$3 '$; LVFKHDSDQG(XUR6WR[[LV «DQG«PDUNHWYRODWLOLW\LV«DQGRXU 3RUWIROLRLVVKRUWRI«DQGXQHPSOR\PHQWUDWH LVJRLQJGRZQDQG«ZKDW""

7KHSUREOHP VRUU\,GRQ¶WKDYHWLPHWRZDLWDQKRXUIRUP\ UHVXOWV,QHHGWKHPLQVHFRQGV FUD]\P\EDFNWHVWLQJZDVSRVLWLYH¼¼¼ ZHKDYHWRWHVWWKLVVWUDWHJ\LQUHDOWLPH QRZ

7KHSUREOHP Ɣ \HDUVRIILQDQFLDOWLFNGDWD 7% Ɣ YROXPHLVJURZLQJOLNHFUD]\ *%GD\ Ɣ YROXPHLVYHU\³VSLN\´ YRODWLOH
Ɣ ILQGSDWWHUQV Ɣ ³EDFNWHVWLQJ´RIVWUDWHJLHV Ɣ UXQVWUDWHJLHVLQUHDOWLPH

:KDWZHQHHG" Ɣ KLVWRULFDOGDWDVWRUH Ɣ UHDOWLPHGDWDVWRUH WRGD\VGDWD Ɣ H[SUHVVLYHTXHU\ODQJXDJH ż VHOHFWLRQ
ż DJJUHJDWLRQ Ɣ YHU\ VLPSOHHYHQWSURFHVVLQJ 3XE6XE Ɣ SHUIRUPDQFH

T Ɣ GHYHORSHGE\$UWKXU:KLWQH\ Ɣ $3/ V !N V !T V
Ɣ ³YHFWRU´³DUUD\´SURFHVVLQJODQJXDJH Ɣ ³WDEOH´³FROXPQ´RULHQWHG Ɣ .;6\VWHPV Ɣ )LQDQFLDOLQGXVWU\EDFNJURXQG Ɣ ³PRVWO\´VLQJOHWKUHDGHG

¼½¾ R@C<Q@<GDNOJAGJIBN x<x=x> NTH=JGGDNO NPH¼½¾ÕÁ <GGAPI>ODJINRJMFRDOCGDNON ¿Ï¼½¾ÕÀÁÂ @Q<GP<O@NAMJHMDBCOOJG@AO ¾¥¿ÏÀÕ½Â ¼½¾
Ï¿ÀÁ ÕÀÂÄ x<x=x>¼½¾ R@C<Q@<?D>O ¡¢ODH@ NTH KMD>@ R@C<Q@<O<=G@ AGDKxODH@xNTHxKMD>@ O<=G@DNAGDKK@??D>OJAGDNON N@G@>OAMJHORC@M@ODH@Õ¼»»»»»NTHÓx<DHG@M LNLG

NGE Ɣ ³LQPHPRU\GDWDEDVHZLWKSHUVLVWHQW EDFNLQJ´ Ɣ VDPHLQPHPRU\DQGRQGLVNUHSUHVHQWDWLRQ Ɣ FROXPQRULHQWHG Ɣ TXHU\ZLWKT!N!NGE
Ɣ ³RSWLPL]HGGLVNDFFHVV´

NGE 1RVWRUHGSURFHXGUHVYVDSSOLFDWLRQVHUYHU 1R³VSHFLDO´WUDQVSRUWIRUPDWRYHUKHDG 1R250SUREOHPV 1R³XQLYHUVDO´GDWDEDVH

RQGLVN ?=J??½»¼»»¼»¼ ?=@Q@I½»¼»»¼»½ ?=J??½»¼»»¼»¾ ?=@Q@I½»¼»»¼»¿ ?=J??½»¼»»¼»À ?=J??½»¼»»¼¾¼OM<?@ H ODH@
NTH KMD>@ URRW VHJPHQW SDUWLWLRQ GDWH WDEOH FROXPQ N@G@>OKMD>@AMJHOM<?@ RC@M@?<O@ÕÓ½»¼»»¼½» NTHÓx<DHG@M

RQGLVN Ɣ VHJPHQWVDUHUHDGLQSDUDOOHO PXOWLSOH,2 FKDQQHOV Ɣ RQO\UHDGUHTXLUHGSDUWLWLRQV Ɣ RQO\UHDGUHTXLUHGFROXPQV Ɣ
GDWDLVVRUWHGE\V\PWLPH

N@G@>OAMJHH@O@MRC@M@H@O@MD?Ó¼»»»»»»» LNLG ¡xH@O@M@IGDNOÓxH@O@MD?¼»»»»»»» »= ¢ API>ODJI<GN@G@>O N@G@>O<Q<QBPN<B@=TÀ»»»»»»»»»S=<MODH@AMJHH@O@MRC@M@H@O@MD?Ó¼»»»»»»» O»PK?<O@xNºODH@AMJHN@G@>OODH@P»PN<B@AMJHH@O@MRC@M@H@O@MD?Ó¼»»»»»»» O¼PK?<O@xNºODH@AMJHN@G@>OODH@P¼PN<B@AMJHH@O@MRC@M@H@O@MD?Ó¼»»»»»»¼ <E¡xODH@O»O¼¢
ODH@EJDI R»»»»¼»»»»»»»»»¼»»»»»Ï¬O»ODH@ RE¡RxODH@O»O¼<QBxP¼ HDIxP¼ ¢ RDI?JREJDI COOK>J?@FS>JHRNQI>J?@>JJF=JJF>J?@OPOJMD<G

6RXUFHV %RUURU 4)RU0RUWDOV KWWSN[FRPPXQLW\FRP KWWSFRGHN[FRPZLNL KWWSVJURXSVJRRJOHFRPIRUXPIRUXPSHUVRQDONGESOXV KWWSZZZN[FRPTGTKWP KWWSZZZN[FRPTGTKWP
KWWSZZZN[FRPTGNGEKWP KWWSZZZN[FRPTGNGEKWP

0LFKDHO:LWWLJ #KHOORPLFKLE\H JLWKXEFRPPLFKDHOZLWWLJ SRVW#PLFKDHOZLWWLJLQIR

6LPLODU Ɣ KLVWRULFDOGDWDVWRUH ż 0RQHW'%

q and kdb+

q and kdb+

Michael Wittig

More Decks by Michael Wittig

Other Decks in Programming

Featured

Transcript

TDQGNGE IURPYHFWRUSURFHVVLQJWRFROXPQRULHQWHGGDWDEDVH

7KHSUREOHP P\VWUDWHJ\XVXDOO\LVEX\VHOO'DLPOHULI FKHDSFRPSDUHGWR%0:DQG9RONVZDJHQ ZKHQ9:$3 '$; LVFKHDSDQG(XUR6WR[[LV «DQG«PDUNHWYRODWLOLW\LV«DQGRXU 3RUWIROLRLVVKRUWRI«DQGXQHPSOR\PHQWUDWH LVJRLQJGRZQDQG«ZKDW""

7KHSUREOHP VRUU\,GRQ¶WKDYHWLPHWRZDLWDQKRXUIRUP\ UHVXOWV,QHHGWKHPLQVHFRQGV FUD]\P\EDFNWHVWLQJZDVSRVLWLYH¼¼¼ ZHKDYHWRWHVWWKLVVWUDWHJ\LQUHDOWLPH QRZ

7KHSUREOHP Ɣ \HDUVRIILQDQFLDOWLFNGDWD 7% Ɣ YROXPHLVJURZLQJOLNHFUD]\ *%GD\ Ɣ YROXPHLVYHU\³VSLN\´ YRODWLOH

:KDWZHQHHG" Ɣ KLVWRULFDOGDWDVWRUH Ɣ UHDOWLPHGDWDVWRUH WRGD\VGDWD Ɣ H[SUHVVLYHTXHU\ODQJXDJH ż VHOHFWLRQ

T Ɣ GHYHORSHGE\$UWKXU:KLWQH\ Ɣ $3/ V !N V !T V

¼½¾ R@C<Q@<GDNOJAGJIBN x<x=x> NTH=JGGDNO NPH¼½¾ÕÁ <GGAPI>ODJINRJMFRDOCGDNON ¿Ï¼½¾ÕÀÁÂ @Q<GP<O@NAMJHMDBCOOJG@AO ¾¥¿ÏÀÕ½Â ¼½¾

NGE Ɣ ³LQPHPRU\GDWDEDVHZLWKSHUVLVWHQW EDFNLQJ´ Ɣ VDPHLQPHPRU\DQGRQGLVNUHSUHVHQWDWLRQ Ɣ FROXPQRULHQWHG Ɣ TXHU\ZLWKT!N!NGE

NGE 1RVWRUHGSURFHXGUHVYVDSSOLFDWLRQVHUYHU 1R³VSHFLDO´WUDQVSRUWIRUPDWRYHUKHDG 1R250SUREOHPV 1R³XQLYHUVDO´GDWDEDVH

RQGLVN ?=J??½»¼»»¼»¼ ?=@Q@I½»¼»»¼»½ ?=J??½»¼»»¼»¾ ?=@Q@I½»¼»»¼»¿ ?=J??½»¼»»¼»À ?=J??½»¼»»¼¾¼OM<?@ H ODH@

RQGLVN Ɣ VHJPHQWVDUHUHDGLQSDUDOOHO PXOWLSOH,2 FKDQQHOV Ɣ RQO\UHDGUHTXLUHGSDUWLWLRQV Ɣ RQO\UHDGUHTXLUHGFROXPQV Ɣ

6RXUFHV %RUURU 4)RU0RUWDOV KWWSN[FRPPXQLW\FRP KWWSFRGHN[FRPZLNL KWWSVJURXSVJRRJOHFRPIRUXPIRUXPSHUVRQDONGESOXV KWWSZZZN[FRPTGTKWP KWWSZZZN[FRPTGTKWP

0LFKDHO:LWWLJ #KHOORPLFKLE\H JLWKXEFRPPLFKDHOZLWWLJ SRVW#PLFKDHOZLWWLJLQIR

6LPLODU Ɣ KLVWRULFDOGDWDVWRUH ż 0RQHW'%