Upgrade to Pro — share decks privately, control downloads, hide ads and more …

A whirlwind tour through Lingual: ANSI SQL for Apache Hadoop

6da3d4048a89eae74e790545d08ff687?s=47 André Kelpe
November 14, 2013
Programming
1
160

A whirlwind tour through Lingual: ANSI SQL for Apache Hadoop

Slides from my #devoxx 2013 talk about Cascading Lingual

6da3d4048a89eae74e790545d08ff687?s=128

André Kelpe

November 14, 2013
Tweet

More Decks by André Kelpe

See All by André Kelpe
6da3d4048a89eae74e790545d08ff687?s=48 fs111
0
130
6da3d4048a89eae74e790545d08ff687?s=48 fs111
1
190
6da3d4048a89eae74e790545d08ff687?s=48 fs111
0
170
6da3d4048a89eae74e790545d08ff687?s=48 fs111
0
140
6da3d4048a89eae74e790545d08ff687?s=48 fs111
2
5k

Other Decks in Programming

See All in Programming
7afd74181738991f61d6061562ff419d?s=48 hitsuji_haneta
2
670
9516a84fcc28d6456d4fc87a5c8b130f?s=48 kidel
0
1.2k
38b49a5fa7e0519f2919ac342a463980?s=48 marcobrador
0
240
9fdb3aa57e5fb99b3772976b0b903e53?s=48 chikuwait
8
1.8k
8d7983d03db1fb756ccdbcbf7b760f98?s=48 yamada1826
0
330
Be33a704f251c77162896ac5ba03e9ac?s=48 azuki
0
210
3499a1d71fa70b8ee44816ca9e7329fe?s=48 bells17
1
410
F0012066f4fad79deb33bf5e9799b557?s=48 francescostrazzullo
0
110
7437b838338581b08ca72340b05475b9?s=48 abt
1
260
1d3fab5abb85de9abc092b72a036965b?s=48 bot6rice
1
160
744f1c2c6cbea2ff5104b0ac512936bd?s=48 hpgrahsl
0
110
F86d970e452f844b2849522cf3cc7e5f?s=48 hedgehognoodle
2
200

Featured

See All Featured
1b75d89772b7eea1f6c89fbf362607d9?s=48 moore
128
21k
A60068bce2e73de3a37ca9d2dbe36092?s=48 bryan
104
11k
8ec1383b240b5ba15ffb9743fceb3c0e?s=48 philnash
20
900
24fc194843a71f10949be18d5a692682?s=48 gr2m
85
11k
F716e5f737fcfb03f2cf8d1a184f1dbe?s=48 nonsquared
83
3.6k
E43f8dfd01057f8304f5f8fb9a294d7d?s=48 jeffersonlam
333
16k
A9179349dd2bdc67f377719f56d85656?s=48 garrettdimon
291
110k
D47656e20ff5e42f125fc5ea0fd636c6?s=48 tmm1
64
10k
78b475797a14c84799063c7cd073962f?s=48 holman
450
140k
9128d500301ae51524e887bb680f471d?s=48 caitiem20
314
18k
88bc30284c6a424b63a92aad27d0ba36?s=48 jnunemaker
PRO
43
4.8k
170b760e0147792d0140e59ec78e9893?s=48 eitanlees
118
10k

Transcript

  1. @fs111 #DV13 #lingual A whirlwind tour through Lingual: ANSI SQL

    for Apache Hadoop André Kelpe concurrentinc.com

  2. @fs111 #DV13 #lingual Speaker André Kelpe Software Engineer at Concurrent

    The company behind Cascading and Lingual concurrentinc.com / @concurrent andre@concurrentinc.com @fs111

  3. @fs111 #DV13 #lingual Agenda Cascading and Lingual Lingual: design goals

    Lingual: features Demo: Lingual in action Q&A

  4. @fs111 #DV13 #lingual Cascading http://cascading.org

  5. @fs111 #DV13 #lingual Cascading terminology Taps are sources and sinks

    for data Schemes represent the format of the data Pipes are connecting Taps

  6. @fs111 #DV13 #lingual Cascading terminology Tuples flow through Pipes Fields

    describe the Tuples Operations are executed on Tuples in TupleStreams Flows get scheduled and executed

  7. @fs111 #DV13 #lingual Not just Java!

  8. @fs111 #DV13 #lingual Scalding Scala DSL on top of Cascading

    Developed by twitter https://github.com/twitter/scalding/

  9. @fs111 #DV13 #lingual Cascalog Clojure DSL on top of Cascading

    Inspired by datalog https://github.com/nathanmarz/cascalog

  10. @fs111 #DV13 #lingual ANSI SQL via Lingual http://www.cascading.org/lingual/

  11. @fs111 #DV13 #lingual Lingual – design goals 1/3 Immediate Data

    Access SQL access via Shell or JDBC driver

  12. @fs111 #DV13 #lingual Lingual – design goals 2/3 Simplify SQL

    Migration move SQL workflows on your Hadoop Cluster via Cascading flows or JDBC driver

  13. @fs111 #DV13 #lingual Lingual – design goals 3/3 Simplify System

    & Data Integration Read and write from hdfs, jdbc, memcached, HBase, redshift...

  14. @fs111 #DV13 #lingual Lingual – ANSI SQL on Cascading http://www.cascading.org/lingual/

  15. @fs111 #DV13 #lingual Lingual and Cascading are about batch processing

    large amounts of data

  16. @fs111 #DV13 #lingual Demo

  17. @fs111 #DV13 #lingual Lingual – ANSI SQL on Cascading http://www.cascading.org/lingual/

  18. @fs111 #DV13 #lingual Q&A @fs111 / http://concurrentinc.com http://cascading.org/lingual

  19. @fs111 #DV13 #lingual Link collection http://www.cascading.org/lingual/ http://www.cascading.org/ http://docs.cascading.org/lingual/1.0/ https://github.com/Cascading/ http://concurrentinc.com

    https://groups.google.com/forum/#!forum/lingual-user https://groups.google.com/forum/#!forum/cascading-user http://docs.cascading.org/impatient/ https://github.com/Cascading/vagrant-cascading-hadoop-cluster