Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Chasing the Elephant

Chasing the Elephant

JRubyConfEU 2013

8c21306523b16ba5dd35c3549bf90994?s=128

Theo Hultberg

August 15, 2013
Tweet

Transcript

  1. @iconara CHASING THE ELEPHANT

  2. Theo / @iconara

  3. chief architect at BURT

  4. big data analytics with JRuby

  5. RUBY

  6. RUBY

  7. RUBY JRUBY

  8. JRUBY IS AWESOME BECAUSE RUBY IS GREAT, AND THE JVM

    IS GREAT
  9. hot_bunnies, eurydice, multimeter, mikka, msgpack-jruby

  10. None
  11. HADOOP

  12. JRUBY SUPERCOMPUTING 40 TiB data, 120 EC2 cc2.8xlarge, 1920 cores,

    7260 GiB RAM
  13. JAVA ALL THE WAY DOWN

  14. HADOOP STREAMING Wukong, Dumbo

  15. RUBYDOOP rubydoop.org + =

  16. None
  17. None
  18. main()

  19. Class.forName("...")

  20. Class.forName("...")

  21. None
  22. module WordCount class Mapper def map(key, value, context) value.to_s.downcase.split.each do

    |word| key = Hadoop::Io::Text.new(word) value = Hadoop::Io::IntWritable.new(1) context.write(key, value) end end end end
  23. RUBYDOOP IS LOW LEVEL I would love to see someone

    write something like Scalding or Cascading on top of it
  24. RUBYDOOP rubydoop.org

  25. RUBYDOOP rubydoop.org v1.1.0

  26. KTHXBAI @iconara github.com/iconara architecturalatrocities.com burtcorp.com