@fs111
#DV13 #lingual
A whirlwind tour through
Lingual: ANSI SQL for Apache
Hadoop
André Kelpe
concurrentinc.com
Slide 2
Slide 2 text
@fs111
#DV13 #lingual
Speaker
André Kelpe
Software Engineer at Concurrent
The company behind Cascading and Lingual
concurrentinc.com / @concurrent
[email protected]
@fs111
Slide 3
Slide 3 text
@fs111
#DV13 #lingual
Agenda
Cascading and Lingual
Lingual: design goals
Lingual: features
Demo: Lingual in action
Q&A
@fs111
#DV13 #lingual
Cascading terminology
Taps are sources and sinks for data
Schemes represent the format of the data
Pipes are connecting Taps
Slide 6
Slide 6 text
@fs111
#DV13 #lingual
Cascading terminology
Tuples flow through Pipes
Fields describe the Tuples
Operations are executed on Tuples in
TupleStreams
Flows get scheduled and executed
Slide 7
Slide 7 text
@fs111
#DV13 #lingual
Not just Java!
Slide 8
Slide 8 text
@fs111
#DV13 #lingual
Scalding
Scala DSL on top of Cascading
Developed by twitter
https://github.com/twitter/scalding/
Slide 9
Slide 9 text
@fs111
#DV13 #lingual
Cascalog
Clojure DSL on top of Cascading
Inspired by datalog
https://github.com/nathanmarz/cascalog
Slide 10
Slide 10 text
@fs111
#DV13 #lingual
ANSI SQL via Lingual
http://www.cascading.org/lingual/
Slide 11
Slide 11 text
@fs111
#DV13 #lingual
Lingual – design goals 1/3
Immediate Data Access
SQL access via Shell or JDBC driver
Slide 12
Slide 12 text
@fs111
#DV13 #lingual
Lingual – design goals 2/3
Simplify SQL Migration
move SQL workflows on your Hadoop Cluster
via Cascading flows or JDBC driver
Slide 13
Slide 13 text
@fs111
#DV13 #lingual
Lingual – design goals 3/3
Simplify System
& Data Integration
Read and write from hdfs, jdbc, memcached,
HBase, redshift...
Slide 14
Slide 14 text
@fs111
#DV13 #lingual
Lingual – ANSI SQL on Cascading
http://www.cascading.org/lingual/
Slide 15
Slide 15 text
@fs111
#DV13 #lingual
Lingual and Cascading are
about batch processing large
amounts of data
Slide 16
Slide 16 text
@fs111
#DV13 #lingual
Demo
Slide 17
Slide 17 text
@fs111
#DV13 #lingual
Lingual – ANSI SQL on Cascading
http://www.cascading.org/lingual/