language • Used to analyse large data sets • Used to create MapReduce jobs • Abstracts definition of jobs • Uses Pig Latin to define jobs • Less code needed • Compiles to MapReduce code www.semtech-solutions.co.nz [email protected]
to use it – Grunt – Pig's interactive shell – Write Pig Latin in a script file – Embed Pig commands in another language • Run modes – Local mode – single machine – Hadoop – run on a Hadoop/MapReduce cluster • Creates MapReduce code automatically www.semtech-solutions.co.nz [email protected]
• It is data omnivorous • It is easy to learn • It is widely used • Minor performance loss – Compared to native code • It can be extended via user defined functions ( UDF ) www.semtech-solutions.co.nz [email protected]
AS (line:chararray); -- Extract words from each line and put them into a pig bag -- datatype, then flatten the bag to get one word on each row words = FOREACH input_lines GENERATE FLATTEN(TOKENIZE(line)) AS word; -- filter out any words that are just white spaces filtered_words = FILTER words BY word MATCHES '\\w+'; -- create a group for each word word_groups = GROUP filtered_words BY word; -- count the entries in each group word_count = FOREACH word_groups GENERATE COUNT(filtered_words) AS count, group AS word; -- order the records by count ordered_word_count = ORDER word_count BY count DESC; STORE ordered_word_count INTO '/tmp/number-of-words-on-internet'; www.semtech-solutions.co.nz [email protected]
www.semtech-solutions.co.nz – [email protected] • We offer IT project consultancy • We are happy to hear about your problems • You can just pay for those hours that you need • To solve your problems
semtechs
slsops
zozotech
nutslove
genda
tebiki
sansan33
torounit
muziyoshiz
oracle4engineer
dugsong
katarinadahlin
inesmontani
techseoconnect
garrettdimon
mclloyd
brad_frost
caitiem20
addyosmani
mojombo
bkeepers
