Upgrade to Pro — share decks privately, control downloads, hide ads and more …

A whirlwind tour through Lingual: ANSI SQL for Apache Hadoop

6da3d4048a89eae74e790545d08ff687?s=47 André Kelpe
November 14, 2013
Programming
1
160

A whirlwind tour through Lingual: ANSI SQL for Apache Hadoop

Slides from my #devoxx 2013 talk about Cascading Lingual

6da3d4048a89eae74e790545d08ff687?s=128

André Kelpe

November 14, 2013
Tweet

More Decks by André Kelpe

See All by André Kelpe
6da3d4048a89eae74e790545d08ff687?s=48 fs111
0
130
6da3d4048a89eae74e790545d08ff687?s=48 fs111
1
200
6da3d4048a89eae74e790545d08ff687?s=48 fs111
0
180
6da3d4048a89eae74e790545d08ff687?s=48 fs111
0
140
6da3d4048a89eae74e790545d08ff687?s=48 fs111
2
5.1k

Other Decks in Programming

See All in Programming
897f2389f9228d4ef123f4196e75f3a4?s=48 yukimasa123
0
170
91b03a57e0ef56fee59fbddc66a7f6fd?s=48 takasehideki
0
150
Fb0a40e5e7a9dc85520f1646fd41eb44?s=48 higuuu
1
180
461346ac617a22f5bf6346e28d7c7202?s=48 bassbone
0
270
6dd0483f1353a4a359e92633cfd65c64?s=48 wasabeef
2
1.3k
96466365a9bf7e66990f2b9ba547b02f?s=48 yumu19
1
770
E10ba38646a905c8cd2c893c84d5c520?s=48 suzushin54
0
280
07417ad42eeae4f8233038000f990f05?s=48 chirurimu
0
310
3799f16d41d627cf75b984212844c7ed?s=48 ryamakuchi
1
270
4bcb32bcb73d1ead993a6ad55b189e9e?s=48 rluisr
1
130
7e56a691cfb413e0f6e631791f33fe54?s=48 mismith0227
0
200
Bb4943c6ae1f841c22da2a2142fd8b49?s=48 umireon
0
120

Featured

See All Featured
20bfe76b3d6105641f879fe45cfc9272?s=48 bkeepers
PRO
415
59k
F383c6a4dc55e331bbe2987b622cee6b?s=48 stephaniewalter
267
11k
5f2da528927a2ec9ba4fec2069cbc958?s=48 kneath
225
15k
96270e4c3e5e9806cf7245475c00b275?s=48 addyosmani
1352
200k
9952dfcd5d338f8a8e7175c8a8f65fb5?s=48 wjessup
342
16k
5f50f94346fbcb23706f0707169317a4?s=48 aarron
262
37k
9fa239b3154a0169d99ab7bf1422dd12?s=48 marcelosomers
225
15k
A0517b08532aec8ab2aae3f65d40ad86?s=48 hannesfritz
32
1.2k
3ab1249be442027903e1180025340b3f?s=48 reverentgeek
37
3k
282d599b414022eba5096510f675b6e2?s=48 chrislema
232
16k
5b9fe87ec1faa67a4599782930f45ec9?s=48 sstephenson
149
12k
C0b42577cfc2b321be3474618107e933?s=48 samanthasiow
62
7k

Transcript

  1. @fs111 #DV13 #lingual A whirlwind tour through Lingual: ANSI SQL

    for Apache Hadoop André Kelpe concurrentinc.com

  2. @fs111 #DV13 #lingual Speaker André Kelpe Software Engineer at Concurrent

    The company behind Cascading and Lingual concurrentinc.com / @concurrent andre@concurrentinc.com @fs111

  3. @fs111 #DV13 #lingual Agenda Cascading and Lingual Lingual: design goals

    Lingual: features Demo: Lingual in action Q&A

  4. @fs111 #DV13 #lingual Cascading http://cascading.org

  5. @fs111 #DV13 #lingual Cascading terminology Taps are sources and sinks

    for data Schemes represent the format of the data Pipes are connecting Taps

  6. @fs111 #DV13 #lingual Cascading terminology Tuples flow through Pipes Fields

    describe the Tuples Operations are executed on Tuples in TupleStreams Flows get scheduled and executed

  7. @fs111 #DV13 #lingual Not just Java!

  8. @fs111 #DV13 #lingual Scalding Scala DSL on top of Cascading

    Developed by twitter https://github.com/twitter/scalding/

  9. @fs111 #DV13 #lingual Cascalog Clojure DSL on top of Cascading

    Inspired by datalog https://github.com/nathanmarz/cascalog

  10. @fs111 #DV13 #lingual ANSI SQL via Lingual http://www.cascading.org/lingual/

  11. @fs111 #DV13 #lingual Lingual – design goals 1/3 Immediate Data

    Access SQL access via Shell or JDBC driver

  12. @fs111 #DV13 #lingual Lingual – design goals 2/3 Simplify SQL

    Migration move SQL workflows on your Hadoop Cluster via Cascading flows or JDBC driver

  13. @fs111 #DV13 #lingual Lingual – design goals 3/3 Simplify System

    & Data Integration Read and write from hdfs, jdbc, memcached, HBase, redshift...

  14. @fs111 #DV13 #lingual Lingual – ANSI SQL on Cascading http://www.cascading.org/lingual/

  15. @fs111 #DV13 #lingual Lingual and Cascading are about batch processing

    large amounts of data

  16. @fs111 #DV13 #lingual Demo

  17. @fs111 #DV13 #lingual Lingual – ANSI SQL on Cascading http://www.cascading.org/lingual/

  18. @fs111 #DV13 #lingual Q&A @fs111 / http://concurrentinc.com http://cascading.org/lingual

  19. @fs111 #DV13 #lingual Link collection http://www.cascading.org/lingual/ http://www.cascading.org/ http://docs.cascading.org/lingual/1.0/ https://github.com/Cascading/ http://concurrentinc.com

    https://groups.google.com/forum/#!forum/lingual-user https://groups.google.com/forum/#!forum/cascading-user http://docs.cascading.org/impatient/ https://github.com/Cascading/vagrant-cascading-hadoop-cluster