$30 off During Our Annual Pro Sale. View Details »

Do Flink on Web with FLOW

eastcirclek
October 08, 2019
Programming
0
390

Do Flink on Web with FLOW

We present a web service named FLOW to let users do FLink On Web. FLOW aims to minimize the effort of handwriting streaming applications similar in spirit to Hortonworks Stream Analytics Manager, StreamAnalytix, and Nussknacker by letting users drag and drop graphical icons representing streaming operators on GUI.

FLOW builds on Flink Table API and lets users assemble graphical icons associated with not only basic SQL operations but also advanced SQL operations like window aggregation, temporal join, and pattern recognition (MATCH_RECOGNIZE clause). Its data preview function enables to observe how sample data changes before and after applying each operation on screen. In addition, FLOW shows the sample data as time-series charts and geographical maps by interacting with Elasticsearch and Kibana. Therefore, domain experts with basic knowledge of SQL can design their streaming applications easily on GUI without understanding of Flink DataStream API and Flink CEP library.

In this talk, we first present what motivates the development of FLOW, then show how FLOW can be used to figure out the "Popular Places" exercise in its own style, and lastly explain how FLOW leverages Flink Table API.

eastcirclek

October 08, 2019
Tweet

More Decks by eastcirclek

See All by eastcirclek
Real-time driving score service using Flink
eastcirclek
0
40

Other Decks in Programming

See All in Programming
4. 서비스 플랫폼 레이어 다중화
kakao
PRO
0
140
CSS Grid・Flexboxの 最近の進化とミライ
tonkotsuboy_com
2
690
브런치 FE 심폐소생술
kakao
PRO
1
100
Dynamic VM Scheduling
 in OpenStack
pyama86
1
430
Excelを扱うRubyGemまとめ 2022
ktam1219
0
300
GDG DevFest2022 Songdo - KMM(Kotlin Multiplatform Mobile)
taehwandev
0
240
AWS Glueではじめるデータレイク
marchin1989
0
180
フロントエンドエコシステムで効率化する組織開発
okmttdhr
4
6.3k
JavaScript Testing Framework: Under the Hood(JSConfJP2022)
yoshiaki_togami
0
2.5k
RubyWorld Conference 2022 に初めて現地参加してみて
shiorin
0
170
API 테스트 주도 개발의 시작
kakao
PRO
0
460
Hive Distributed Profiling System in Treasure Data - English version #tdtechtalk
okumin
0
150

Featured

See All Featured
Debugging Ruby Performance
tmm1
67
11k
XXLCSS - How to scale CSS and keep your sanity
sugarenia
236
1.1M
Building Your Own Lightsaber
phodgson
96
4.8k
What's in a price? How to price your products and services
michaelherold
231
9.6k
How to Ace a Technical Interview
jacobian
269
21k
I Don’t Have Time: Getting Over the Fear to Launch Your Podcast
jcasabona
13
1.1k
Building Better People: How to give real-time feedback that sticks.
wjessup
344
17k
JazzCon 2018 Closing Keynote - Leadership for the Reluctant Leader
reverentgeek
175
8.9k
How STYLIGHT went responsive
nonsquared
87
4.1k
In The Pink: A Labor of Love
frogandcode
132
21k
"I'm Feeling Lucky" - Building Great Search Experiences for Today's Users (#IAC19)
danielanewman
215
21k
Optimizing for Happiness
mojombo
364
64k

Transcript

  1. %P'MJOL PO8FC XJUI'-08 %POHXPO,JN FBTUDJSDMFL!TLDPN )BFNFF 1BSL IBFNFFQBSL!TLDPN

  2. T A B L E O F C O N

    T E N T S %P'MJOL PO8FCXJUI'-08  .PUJWBUJPO  %FNP  "SDIJUFDUVSF  4VQQPSUFEPQFSBUJPOTDPOOFDUPST  4VNNBSZ

  3. 4,5'MJOL'PSXBSEUBMLT 'MJOL'PSXBSE 1SFEJDUJWF.BJOUFOBODF XJUI 'MJOL 'MJOL'PSXBSE 3FBMUJNFESJWJOHTDPSF TFSWJDFVTJOH'MJOL

  4. )PXXFVTFEUPXPSL 'MJOL'PSXBSE 1SFEJDUJWF.BJOUFOBODF XJUI 'MJOL 'MJOL'PSXBSE 3FBMUJNFESJWJOHTDPSF TFSWJDFVTJOH'MJOL %PNBJOFYQFSUT 5BSHFUTZTUFNT

    3FRVJSFNFOU 3FGJOFSZFOHJOFFST &YQFOTJWFSFGJOFSZFRVJQNFOU %SJWJOHTDPSFTFSWJDF (FOFSBUJOHBMBSNT"4"1 JOSFBMUJNF (FOFSBUJOHTDPSFT"4"1 JOSFBMUJNF .PCJMJUZTFSWJDFQMBOOFST

  5. /FFETGPSSFBMUJNFTUSFBNQSPDFTTJOH 'MJOL'PSXBSE 1SFEJDUJWF.BJOUFOBODF XJUI 'MJOL 'MJOL'PSXBSE 3FBMUJNFESJWJOHTDPSF TFSWJDFVTJOH'MJOL "QBDIF'MJOL 42-5BCMF"1*

    %BUB4USFBN"1* 1SPDFTT'VODUJPO EZOBNJDUBCMFT TUSFBNT XJOEPXT FWFOUT TUBUF UJNF %PNBJOFYQFSUT 3FGJOFSZFOHJOFFST .PCJMJUZTFSWJDFQMBOOFST )PXUPQSPDFTT SFBMUJNF TUSFBNEBUB

  6. /POOFHMJHJCMFEJTUBODFCFUXFFOEPNBJOFYQFSUTBOE'MJOL 'MJOL'PSXBSE 1SFEJDUJWF.BJOUFOBODF XJUI 'MJOL 'MJOL'PSXBSE 3FBMUJNFESJWJOHTDPSF TFSWJDFVTJOH'MJOL "QBDIF'MJOL 42-5BCMF"1*

    %BUB4USFBN"1* 1SPDFTT'VODUJPO EZOBNJDUBCMFT TUSFBNT XJOEPXT FWFOUT TUBUF UJNF +7.MBOHVBHFT *%& 1SPKFDUNBOBHFNFOU %BUBWJTVBMJ[BUJPO %PNBJOFYQFSUT 3FGJOFSZFOHJOFFST .PCJMJUZTFSWJDFQMBOOFST NJMMJPO LJMPNFUFST   LJMPNFUFST

  7. %BUBFOHJOFFSTCSJEHFUIFHBQ 'MJOL'PSXBSE 1SFEJDUJWF.BJOUFOBODF XJUI 'MJOL 'MJOL'PSXBSE 3FBMUJNFESJWJOHTDPSF TFSWJDFVTJOH'MJOL "QBDIF'MJOL 42-5BCMF"1*

    %BUB4USFBN"1* 1SPDFTT'VODUJPO EZOBNJDUBCMFT TUSFBNT XJOEPXT FWFOUT TUBUF UJNF +7.MBOHVBHFT *%& 1SPKFDUNBOBHFNFOU %BUBWJTVBMJ[BUJPO %BUBFOHJOFFST %PNBJOFYQFSUT 3FGJOFSZFOHJOFFST .PCJMJUZTFSWJDFQMBOOFST

  8. %BUBFOHJOFFSTCSJEHFUIFHBQ 'MJOL'PSXBSE 1SFEJDUJWF.BJOUFOBODF XJUI 'MJOL 'MJOL'PSXBSE 3FBMUJNFESJWJOHTDPSF TFSWJDFVTJOH'MJOL "QBDIF'MJOL 42-5BCMF"1*

    %BUB4USFBN"1* 1SPDFTT'VODUJPO EZOBNJDUBCMFT TUSFBNT XJOEPXT FWFOUT TUBUF UJNF +7.MBOHVBHFT *%& 1SPKFDUNBOBHFNFOU %BUBWJTVBMJ[BUJPO %BUBFOHJOFFST %PNBJOFYQFSUT 3FGJOFSZFOHJOFFST .PCJMJUZTFSWJDFQMBOOFST %BUB %PNBJO LOPXMFEHF 3FRVJSFNFOU %BUB %PNBJO LOPXMFEHF 3FRVJSFNFOU 5SBJOJOH5SBOTGFS .BJOUFOBODF .BJOUFOBODF 5SBJOJOH5SBOTGFS 7FSZJOFGGJDJFOU

  9. -FUEPNBJOFYQFSUTEP'MJOL EJSFDUMZWJB'-08 "QBDIF'MJOL 42-5BCMF"1* %BUB4USFBN"1* 1SPDFTT'VODUJPO EZOBNJDUBCMFT TUSFBNT XJOEPXT FWFOUT

    TUBUF UJNF +7.MBOHVBHFT *%& 1SPKFDUNBOBHFNFOU %BUBWJTVBMJ[BUJPO 7BSJPVTEPNBJOTPG4, .PCJMJUZ *P5 5FMDP &OFSHZ 4FNJDPOEVDUPS &DPNNFSDF .FEJB

  10. '-08 "CTUSBDUJPOMBZFSUPIJEF UIFEFUBJMTPG'MJOL BQQEFWFMPQNFOU -FUEPNBJOFYQFSUTEP'MJOL EJSFDUMZWJB'-08 "QBDIF'MJOL 42-5BCMF"1* %BUB4USFBN"1* 1SPDFTT'VODUJPO

    EZOBNJDUBCMFT TUSFBNT XJOEPXT FWFOUT TUBUF UJNF +7.MBOHVBHFT *%& 1SPKFDUNBOBHFNFOU %BUBWJTVBMJ[BUJPO 7BSJPVTEPNBJOTPG4, .PCJMJUZ *P5 5FMDP &OFSHZ 4FNJDPOEVDUPS &DPNNFSDF .FEJB (SBQIJDBMVTFSJOUFSGBDF UPCVJME TUSFBNQSPDFTTJOHQJQFMJOFTPG 42-PQFSBUJPOT BOEDPOOFDUPST %BUB %PNBJO LOPXMFEHF 3FRVJSFNFOU 42- 42- 42- 42-

  11. T A B L E O F C O N

    T E N T S %P'MJOL PO8FCXJUI'-08  .PUJWBUJPO  %FNPm JEFOUJGZJOHQPQVMBSQMBDFTXJUI'-08  "SDIJUFDUVSF  4VQQPSUFEPQFSBUJPOTDPOOFDUPST  4VNNBSZ

  12. %FNP4DFOBSJP*EFOUJGZJOHQPQVMBSQMBDFTJO/:$ ,BGLB 3JEF*E  5BYJ*E  %SJWFS*E  *T4UBSU 5SVF

    4UBSU$PPSE   &OE$PPSE   1BTTFOHFS$PVOU  &WFOU5JNF  3JEF*E  5BYJ*E  %SJWFS*E  *T4UBSU 5SVF 4UBSU$PPSE   &OE$PPSE   1BTTFOHFS$PVOU  &WFOU5JNF  3JEF*E  5BYJ*E  %SJWFS*E  *T4UBSU 5SVF 4UBSU$PPSE   &OE$PPSE   1BTTFOHFS$PVOU  &WFOU5JNF  3JEF*E  5BYJ*E  %SJWFS*E  *T4UBSU 5SVF 4UBSU$PPSE   &OE$PPSE   1BTTFOHFS$PVOU  &WFOU5JNF  3JEF*E  5BYJ*E  %SJWFS*E  *T4UBSU 5SVF 4UBSU$PPSE   &OE$PPSE   1BTTFOHFS$PVOU  &WFOU5JNF  3JEF*E  5BYJ*E  %SJWFS*E  *T4UBSU 5SVF 4UBSU$PPSE   &OE$PPSE   1BTTFOHFS$PVOU  &WFOU5JNF                   

  13. %FNP4DFOBSJP*EFOUJGZJOHQPQVMBSQMBDFTJO/:$ ,BGLB        

              3FNPWJOH FWFOUTUIBUEPOPU TUBSUPSFOEJO/FX:PSL$JUZ

  14.          

         %FNP4DFOBSJP*EFOUJGZJOHQPQVMBSQMBDFTJO/:$ ,BGLB N N .BQQJOHDPPSEJOBUFTPGFBDI SFDPSE JOUPBHSJEDFMM               

  15. %FNP4DFOBSJP*EFOUJGZJOHQPQVMBSQMBDFTJO/:$ ,BGLB $SFBUJOHBTMJEJOHXJOEPXPGTJ[F NJOVUFT UIBUTMJEFTCZNJOVUFT     

             

  16. %FNP4DFOBSJP*EFOUJGZJOHQPQVMBSQMBDFTJO/:$ ,BGLB N N    $PVOUJOH UIFOVNCFSPGFWFOUT JOFBDIHSJEDFMMQFSXJOEPX

                  

  17. N N %FNP4DFOBSJP*EFOUJGZJOHQPQVMBSQMBDFTJO/:$ ,BGLB  *EFOUJGZJOHUIFDFMMTXIPTFDPVOUJT PSNPSFBTBQPQVMBSQMBDF   

                 
  18. None
  19. None
  20. None
  21. None
  22. None
  23. None
  24. None
  25. None
  26. None
  27. None
  28. None
  29. None
  30. None
  31. None
  32. None
  33. None
  34. None
  35. None
  36. None
  37. None
  38. None
  39. None
  40. None
  41. None
  42. None
  43. None
  44. None
  45. None
  46. None
  47. None

  48. 5 min 5 min 5 min

  49. None
  50. None
  51. None
  52. None
  53. None
  54. None
  55. None
  56. None
  57. None

  58. T A B L E O F C O N

    T E N T S %P'MJOL PO8FCXJUI'-08  .PUJWBUJPO  %FNP  "SDIJUFDUVSFm IPX'-08 JOUFSBDUTXJUI'MJOL  4VQQPSUFEPQFSBUJPOTDPOOFDUPST  4VNNBSZ

  59. 0WFSBMMBSDIJUFDUVSFPG'-08 3&45GVM8FC4FSWFS CZ4QSJOH 'SPOUFOE*OUFSGBDF CZ"OHVMBSKT 4QFD 4QFD 4DIFNB 1SFWJFX FYQFDUFEUPCFSFUVSOFE

    BTSFTQPOTF 1SFWJFX 4DIFNB OPUDPNQVUFEZFU ,BGLB TPVSDF

  60. 0WFSBMMBSDIJUFDUVSFPG'-08 3&45GVM8FC4FSWFS CZ4QSJOH 'SPOUFOE*OUFSGBDF CZ"OHVMBSKT 1SFWJFX 4DIFNB 4QFD 4QFD 4DIFNB

    1SFWJFX 4DIFNB 1SFWJFX ,BGLB TPVSDF 4QFD 1SFWJFX-PBEFST ,BGLB1SFWJFX -PBEFS ,BGLB DPOTVNFS ,BGLB TPVSDF 4QFD */ 4DIFNB 1SFWJFX 065 ,BGLB TPVSDF 'MJOL1SFWJFX -PBEFS 1BSFOU 4DIFNB 1SFWJFX $IJME 4QFD */ -PDBM'MJOL NJOJDMVTUFS 4DIFNB 1SFWJFX 065

  61. 0WFSBMMBSDIJUFDUVSFPG'-08 3&45GVM8FC4FSWFS CZ4QSJOH 'SPOUFOE*OUFSGBDF CZ"OHVMBSKT 1SFWJFX 4DIFNB 4QFD ,BGLB TPVSDF

    1SFWJFX-PBEFST 4QFD ,BGLB TPVSDF 4DIFNB 1SFWJFX 'JMUFS 4QFD 4QFD CVUBMTPQBSFOUT TDIFNBQSFWJFX ,BGLB1SFWJFX -PBEFS ,BGLB DPOTVNFS ,BGLB TPVSDF 4QFD */ 4DIFNB 1SFWJFX 065 'MJOL1SFWJFX -PBEFS 1BSFOU 4DIFNB 1SFWJFX $IJME 4QFD */ -PDBM'MJOL NJOJDMVTUFS 4DIFNB 1SFWJFX 065 1SFWJFX 4DIFNB OPUDPNQVUFEZFU 'JMUFS OPUPOMZTQFD

  62. -PDBM4USFBN&OWJSPONFOU FOW 'MJOL1SFWJFX-PBEFS 4USFBN5BCMF&OWJSPONFOU U&OW WBMQBSFOU5BCMFU&OWSFHJTUFS5BCMF  ,BGLB TPVSDF 4DIFNB

    1SFWJFX SFHJTUFSBMMLOPXO6%'JOTUBODFT U&OWSFHJTUFS'VODUJPO JT*O/:$ OFX(FP6UJMT*T*O/:$ U&OWSFHJTUFS'VODUJPO UP$FMM*E OFX(FP6UJMTUP$FMM*E U&OWSFHJTUFS'VODUJPO UP$PPSET OFX(FP6UJMT5P$PPSET XXXIBMMPXFFODPTUVNFTDPNBEVMUQJHHZCBDLSJEFPODPTUVNFIUNM '-08 '-08QJHHZCBDLTPO'MJOL GPSTDIFNBQSFWJFX DPNQVUBUJPO WBMFOW4USFBN&YFDVUJPO&OWJSPONFOUDSFBUF-PDBM4USFBN&OWJSPONFOU WBMU&OW4USFBN5BCMF&OWJSPONFOUDSFBUF FOW 'MJOL1SFWJFX -PBEFS -PDBM'MJOL NJOJDMVTUFS 4DIFNB 1SFWJFX 065 WBMUBCMFQBSFOU5BCMFGJMUFS JT*O/:$ TUBSU-PO TUBSU-BU JT*O/:$ TUBSU-PO TUBSU-BU   'JMUFS 4QFD UPHFUUIFSFTVMUJOUIJTUISFBE U&OWUP"QQFOE4USFBN UBCMF 3PXDMBTT BEE4JOL OFX$PMMFDU4JOL FOWFYFDVUF SFUVSOBOE SFUVSO $PMMFDU4JOLWBMVFT UBCMFHFU4DIFNB 4DIFNB 1SFWJFX ,BGLB TPVSDF 4DIFNB 1SFWJFX 'JMUFS 4QFD */ 5BCMFT 'VODUJPOT QBSFOU5BCMF UBCMF JT*O/:$ GMPBU GMPBU ªCPPMFBO UP$FMM*E GMPBU GMPBU ªJOU UP$PPSEJOUª GMPBU GMPBU

  63. ,BGLB TPVSDF 4DIFNB 1SFWJFX 4DIFNB 1SFWJFX 'JMUFS 0WFSBMMBSDIJUFDUVSFPG'-08 3&45GVM8FC4FSWFS CZ4QSJOH

    'SPOUFOE*OUFSGBDF CZ"OHVMBSKT 1SFWJFX 4DIFNB 4QFD ,BGLB TPVSDF 1SFWJFX-PBEFST 4QFD ,BGLB TPVSDF 4DIFNB 1SFWJFX 'JMUFS 4QFD 4DIFNB 1SFWJFX 1SFWJFX 4DIFNB 4QFD 'JMUFS ,BGLB1SFWJFX -PBEFS ,BGLB DPOTVNFS ,BGLB TPVSDF 4QFD */ 4DIFNB 1SFWJFX 065 'MJOL1SFWJFX -PBEFS 1BSFOU 4DIFNB 1SFWJFX $IJME 4QFD */ -PDBM'MJOL NJOJDMVTUFS 4DIFNB 1SFWJFX 065

  64. 0WFSBMMBSDIJUFDUVSFPG'-08 3&45GVM8FC4FSWFS CZ4QSJOH 1SFWJFX 4DIFNB 4QFD ,BGLB TPVSDF 1SFWJFX 4DIFNB

    4QFD 'JMUFS 1SFWJFX 4DIFNB 4QFD 4FMFDU 1SFWJFX 4DIFNB 4QFD 8JOEPX 1SFWJFX 4DIFNB 4QFD 42- RVFSZ 1SFWJFX-PBEFST 4QFD 1BSFOU 4DIFNB 1SFWJFX $IJME 4QFD 4DIFNB 1SFWJFX 'SPOUFOE*OUFSGBDF CZ"OHVMBSKT ,BGLB1SFWJFX -PBEFS ,BGLB DPOTVNFS ,BGLB TPVSDF 4QFD */ 4DIFNB 1SFWJFX 065 'MJOL1SFWJFX -PBEFS 1BSFOU 4DIFNB 1SFWJFX $IJME 4QFD */ -PDBM'MJOL NJOJDMVTUFS 4DIFNB 1SFWJFX 065 1BSFOU 4DIFNB 1SFWJFX 4DIFNB 1SFWJFX $IJME

  65. 0WFSBMMBSDIJUFDUVSFPG'-08 3&45GVM8FC4FSWFS CZ4QSJOH 'SPOUFOE*OUFSGBDF CZ"OHVMBSKT 1SFWJFX 4DIFNB 4QFD ,BGLB TPVSDF

    1SFWJFX 4DIFNB 4QFD 'JMUFS 1SFWJFX 4DIFNB 4QFD 4FMFDU 1SFWJFX 4DIFNB 4QFD 8JOEPX 1SFWJFX 4DIFNB 4QFD 42- RVFSZ 1SFWJFX 4DIFNB 4QFD &44JOL 1SPKFDU(FOFSBUPS CZKBWBQPFUGSFFNBSLFS 4DIFNB 4QFD ,BGLB TPVSDF 4DIFNB 4QFD 'JMUFS 4DIFNB 4QFD 4FMFDU 4DIFNB 4QFD 8JOEPX 4DIFNB 4QFD 42- RVFSZ 4DIFNB 4QFD &44JOL .BWFOQSPKFDU 'MJOLBQQMJDBUJPO

  66. T A B L E O F C O N

    T E N T S %P'MJOL PO8FCXJUI'-08  .PUJWBUJPO  %FNP  "SDIJUFDUVSF  4VQQPSUFEPQFSBUJPOTDPOOFDUPST  4VNNBSZ

  67. 4VQQPSUFEPQFSBUJPOTDPOOFDUPST #BTJD42-PQFSBUJPOT &YUFSOBMDPOOFDUPST "EWBODFE42-PQFSBUJPOT

  68. 1BUUFSOSFDPHOJUJPO ."5$)@3&$0(/*;&DMBVTF

  69. 1BUUFSOSFDPHOJUJPO ."5$)@3&$0(/*;&DMBVTF

  70. 1BUUFSOSFDPHOJUJPO ."5$)@3&$0(/*;&DMBVTF

  71. 1BUUFSOSFDPHOJUJPO ."5$)@3&$0(/*;&DMBVTF

  72. 1BUUFSOSFDPHOJUJPO ."5$)@3&$0(/*;&DMBVTF

  73. 1BUUFSOSFDPHOJUJPO ."5$)@3&$0(/*;&DMBVTF

  74. 1BUUFSOSFDPHOJUJPO ."5$)@3&$0(/*;&DMBVTF

  75. 1BUUFSOSFDPHOJUJPO ."5$)@3&$0(/*;&DMBVTF

  76. 4VQQPSUFEPQFSBUJPOTDPOOFDUPST #BTJD42-PQFSBUJPOT &YUFSOBMDPOOFDUPST "EWBODFE42-PQFSBUJPOT

  77. 5FNQPSBMKPJO

  78. 5FNQPSBMKPJO

  79. 5FNQPSBMKPJO

  80. 5FNQPSBMKPJO

  81. T A B L E O F C O N

    T E N T S %P'MJOL PO8FCXJUI'-08  .PUJWBUJPO  %FNP  "SDIJUFDUVSF  4VQQPSUFEPQFSBUJPOTDPOOFDUPST  4VNNBSZ

  82. 4VNNBSZ $POOFDUPST  ,BGLBTPVSDF  ,BGLBTJOL  &MBTUJDTFBSDITJOL $IBSUCBDLFOE 

    &MBTUJDTFBSDI ,JCBOB  NFUBUSPOEJTDPWFSZ TJTUFSQSPKFDU 42-PQFSBUJPOT  4FMFDU  'JMUFS  8JOEPX  42-RVFSZ  1BUUFSOSFDPHOJUJPO  5FNQPSBMKPJO 1SPKFDUDPEFHFOFSBUJPO  .BWFO  (SBEMF 6%'TVQQPSU  4DBMBSGVODUJPOT  5BCMFGVODUJPOT  "HHSFHBUFGVODUJPOT  5BCMFBHHSFHBUFGVODUJPOT %BUBQSFWJFX  5BCMF  5JNFTFSJFTDIBSU  (FPHSBQIJDBMNBQ 'VUVSFXPSL  .PSFDPOOFDUPSTPQFSBUJPOT  +PCNBOBHFNFOU  #BUDIKPCTVQQPSU