Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
A Riak Query Tale
Search
Mathias Meyer
February 01, 2012
Programming
5
1k
A Riak Query Tale
An introduction to the abundance of ways you can get data out of Riak.
Mathias Meyer
February 01, 2012
Tweet
Share
More Decks by Mathias Meyer
See All by Mathias Meyer
Building and Scaling an Distributed and Inclusive Team
roidrage
0
1.3k
cooking infrastructure with chef
roidrage
4
230
The Message Queue is Dead, Long Live the Message Queue
roidrage
4
700
riak-js
roidrage
1
280
designing for concurrency with riak
roidrage
11
1.8k
metrics, monitoring, logging
roidrage
82
15k
design for cloud - jax 2012
roidrage
2
300
Don't Use NoSQL
roidrage
10
1.1k
Designing Applications for Amazon Web Services (GOTO Aarhus)
roidrage
6
360
Other Decks in Programming
See All in Programming
Git Sync を超える!OSS で実現する CDK Pull 型デプロイ / Deploying CDK with PipeCD in Pull-style
tkikuc
4
350
SQLアンチパターン第2版 データベースプログラミングで陥りがちな失敗とその対策 / Intro to SQL Antipatterns 2nd
twada
PRO
11
1.3k
Hack Claude Code with Claude Code
choplin
7
2.5k
Model Pollution
hschwentner
1
160
20250708_JAWS_opscdk
takuyay0ne
2
130
ご注文の差分はこちらですか? 〜 AWS CDK のいろいろな差分検出と安全なデプロイ
konokenj
3
580
チームで開発し事業を加速するための"良い"設計の考え方 @ サポーターズCoLab 2025-07-08
agatan
1
470
おやつのお供はお決まりですか?@WWDC25 Recap -Japan-\(region).swift
shingangan
0
140
なぜ「共通化」を考え、失敗を繰り返すのか
rinchoku
1
680
たった 1 枚の PHP ファイルで実装する MCP サーバ / MCP Server with Vanilla PHP
okashoi
1
300
Python型ヒント完全ガイド 初心者でも分かる、現代的で実践的な使い方
mickey_kubo
1
240
The Modern View Layer Rails Deserves: A Vision For 2025 And Beyond @ RailsConf 2025, Philadelphia, PA
marcoroth
2
730
Featured
See All Featured
Practical Tips for Bootstrapping Information Extraction Pipelines
honnibal
PRO
21
1.3k
Scaling GitHub
holman
460
140k
Stop Working from a Prison Cell
hatefulcrawdad
271
21k
The MySQL Ecosystem @ GitHub 2015
samlambert
251
13k
The Illustrated Children's Guide to Kubernetes
chrisshort
48
50k
"I'm Feeling Lucky" - Building Great Search Experiences for Today's Users (#IAC19)
danielanewman
229
22k
How STYLIGHT went responsive
nonsquared
100
5.6k
Making the Leap to Tech Lead
cromwellryan
134
9.4k
Product Roadmaps are Hard
iamctodd
PRO
54
11k
Building a Modern Day E-commerce SEO Strategy
aleyda
42
7.4k
Into the Great Unknown - MozCon
thekraken
40
1.9k
Building Adaptive Systems
keathley
43
2.7k
Transcript
A Riak Query Tale Mathias Meyer, @roidrage NoSQL Cologne
http://riakhandbook.com
Riak Distributed Database Fault-Tolerant Content-Agnostic Scalable on Demand
Querying Data
Key-Value $ curl localhost:8098/riak/users/roidrage
Links $ curl -‐v localhost:8098/riak/users/roidrage < HTTP/1.1 200 OK <
Link: </riak/users/klimpong>; riaktag="friend"
Links $ curl .../riak/users/roidrage/users,friend,_/
Listing Keys $ curl .../riak/users?keys=true
Don’t do that!
Streaming Keys $ curl .../riak/users?keys=stream
Avoid that!
Loads all the keys.
MapReduce
MapReduce Transform (Map) Aggregate (Reduce)
Warning: JavaScript
MapReduce riak.add("users"). map("Riak.mapValues").
run()
MapReduce var nameLength = function(value) { var doc
= Riak.mapValues(value)[0]; return [doc.length]; }
MapReduce riak.add("users"). map(nameLength).
run()
MapReduce riak.add("users"). map(nameLength).
reduce("Riak.reduceSum"). run()
MapReduce var average = function(values) { var avg
= values.reduce(function(n, sum) { return sum += n; }, 0); return [(avg / values.length)]; }
MapReduce riak.add("users"). map(nameLength).
reduce(average). run()
MapReduce riak.add("users"). map(nameLength).
reduce(average). run() Uh-Oh!
MapReduce riak.add(["users", "roidrage"]). map(nameLength).
reduce(average). run() Better!
JavaScript M/R Breaks with Millions of Objects Uses External Libraries
Serializes Data for JavaScript
Warning: Erlang
MapReduce riak.add('tweets'). map({language: 'erlang',
module: 'riak_kv_mapreduce', function: 'map_object_value'}).run()
MapReduce $ riak attach > {ok, C} = riak:local_client().
MapReduce C:mapred([{<<"users">>, <<"roidrage">>}], [{map, {modfun, riak_kv_mapreduce, map_object_value}, none, false}, {reduce,
{modfun, riak_kv_mapreduce, reduce_count_inputs}, none, true}]).
MapReduce ExtractFirstName1 = fun(RObject, _, _) -‐>
Value = riak_object:get_value(RObject), [FirstName, _] = re:split(Value, " "), [FirstName] end.
MapReduce C:mapred([{<<"users">>, <<"roidrage">>}],
[{map, {qfun, ExtractFirstName}, none, true}]).
Erlang M/R Much more efficient than JavaScript No serialization No
ad-hoc functions through HTTP
Key-Filters Reduce MapReduce input Based on key matches
Key-Filters riak.add({bucket: 'users', key_filters: [["matches", "^roid"]]})
Key-Filters riak.add({bucket: 'users', key_filters: [["to_upper"],
["matches", "^ROID"]]})
Key-Filters riak.add({bucket: 'users', key_filters: [["to_upper"],
["to_lower"], ["matches", "^roid"]]})
Key-Filters riak.add({bucket: 'users', key_filters: [["to_upper"],
["ends_with", "RAGE"]]})
Key-Filters riak.add({bucket: 'users', key_filters:
[["and", [["string_to_int"], ["less_than", 10]], [["string_to_int"], ["greater_than", 5]]]]})
Don't use key filters.
Riak 2i Sorted Secondary Indexes Simple Reverse Lookups Maintained Manually
Requires LevelDB
Riak 2i curl -‐X PUT .../riak/users/roidrage -‐d @-‐ \
-‐H "Content-‐Type: text/plain" \ -‐H "X-‐Riak-‐Index-‐firstname_bin: mathias" \ -‐H "X-‐Riak-‐Index-‐lastname_bin: meyer"
Riak 2i X-‐Riak-‐Index-‐firstname_bin: Mathias X-‐Riak-‐Index-‐lastname_bin: Meyer
Riak 2i X-‐Riak-‐Index-‐firstname_bin: Mathias X-‐Riak-‐Index-‐lastname_bin: Meyer X-‐Riak-‐Index-‐age_int: 34
Riak 2i X-‐Riak-‐Index-‐firstname_bin: Mathias X-‐Riak-‐Index-‐lastname_bin: Meyer X-‐Riak-‐Index-‐age_int: 34 X-‐Riak-‐Index-‐topics_bin: nosql,cloud,operations
Riak 2i # Match $ curl .../buckets/users/index/firstname_bin/Mathias
Riak 2i # Range $ curl .../buckets/users/index/firstname_bin/Mathias/Till
Riak 2i # Key $ curl .../buckets/users/index/$key/roidrage
Ordered Keys! (sort of)
MapReduce riak.add({bucket: 'users',
index: 'lastname_bin', key: 'mathias'}). map('Riak.mapValuesJson').run()
Riak 2i No Multi-Index Queries Requires Extra Work in the
App Returns only keys Document-partitioned
Riak Search Full-Text Search Solr-ish Interface Integrates with Riak
Riak Search curl -‐X PUT localhost:8098/riak/users -‐d @-‐ \
-‐H "Content-‐Type: application/json" {"props":{"precommit": [{"mod":"riak_search_kv_hook","fun":"precommit"} ]}}
Indexing Riak Objects curl -‐X PUT .../riak/users/roidrage \
-‐d "Mathias Meyer" -‐H "Content-‐Type: text/plain"
Solr-ish Interface curl .../solr/users/select?q=value:Mathias
Riak Search value:Mathias OR value:Till value:Mathias AND value:Meyer value:Mat* value:[Mathias
TO Till]
MapReduce riak.addSearch("users", "value:Mathias"). map("Riak.mapValues").run()
Riak Search Full text search of structured data Term-partitioned Efficient
for one term queries Multiple Interfaces No Anti-Entropy
When?
Key Listings Never! Almost
MapReduce Analytical Queries Fixed Dataset
Key Filters Never!
Riak 2i Simple Lookups and Range Queries Unbounded Queries Full
Fault-Tolerance
Riak Search Larger documents Full indexing Flexible queries Low frequency
terms
Questions?