● Tuples flow through Pipes ● Fields describe the Tuples ● Operations are executed on Tuples in TupleStreams ● Pipes can be merged, spliced, joined etc. ● Pipe-assemblies are reusable components CASCADING TERMINOLOGY 11
FlowConnector uses QueryPlanner to translate FlowDef into Flow to run on computational platform Flows can be orchestrated via Cascade Applications are Directed Acyclic Graphs (DAG) CASCADING TERMINOLOGY 12
COUNTING WORDS (CONT.) 23 // specify a regex operation to split the "document" text lines into a token stream Fields token = new Fields( "token" ); Fields text = new Fields( "text" ); RegexSplitGenerator splitter = new RegexSplitGenerator( token, "[ \\[\\]\\(\\),.]" ); // only returns "token" Pipe docPipe = new Each( "token", text, splitter, Fields.RESULTS ); // determine the word counts Pipe wcPipe = new Pipe( "wc", docPipe ); wcPipe = new GroupBy( wcPipe, token ); wcPipe = new Every( wcPipe, Fields.ALL, new Count(), Fields.ALL ); ...