working, architecture and deployment strategies • To exploit the power of Hadoop to bring out trends, patterns and relations in an available dataset • To be able to apply this technology for other applications in the future Thursday 5 December 2013
of large data sets across clusters of computers • It is designed to scale up from single servers to thousands of machines • Designed to detect and handle failures at the application layer Thursday 5 December 2013
• Sum these counters on Reducer • The benefit from combiners is limited, as it is likely that all pair are distinct • There is no in-memory accumulations Thursday 5 December 2013
and maintain an associative array (“stripe”) where counters for all adjacent items are accumulated • Reducer receives all stripes for leading item i, merges them, and emits the same result as in the Pairs approach • Generates fewer intermediate keys. Hence the framework has less sorting to do. • Greately benefits from combiners. • Performs in-memory accumulation. This can lead to problems, if not properly implemented. • More complex implementation. Thursday 5 December 2013
by and aggregate and emits them. • Reducer receives values to be aggregated already grouped and calculates an aggregation function. • Typical aggregation functions like sum or max can be calculated in a streaming fashion, hence don’t require to handle all values simultaneously • OUR TWEEK : afford the ability to run parameterized dynamic queries by providing command-line arguments during job run Thursday 5 December 2013
nelly_lima
lynnandtonic
minecr
aleyda
jonoalderson
katiecoart
katarinadahlin
emna__ayadi
eileencodes
philnash
smashingmag
cassininazir
