Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Features
Speaker Deck
PRO
Sign in
Sign up for free
Search
Search
A Riak Query Tale
Search
Mathias Meyer
February 01, 2012
Programming
5
1k
A Riak Query Tale
An introduction to the abundance of ways you can get data out of Riak.
Mathias Meyer
February 01, 2012
Tweet
Share
More Decks by Mathias Meyer
See All by Mathias Meyer
Building and Scaling an Distributed and Inclusive Team
roidrage
0
1.3k
cooking infrastructure with chef
roidrage
4
230
The Message Queue is Dead, Long Live the Message Queue
roidrage
4
700
riak-js
roidrage
1
280
designing for concurrency with riak
roidrage
11
1.8k
metrics, monitoring, logging
roidrage
82
15k
design for cloud - jax 2012
roidrage
2
300
Don't Use NoSQL
roidrage
10
1.1k
Designing Applications for Amazon Web Services (GOTO Aarhus)
roidrage
6
360
Other Decks in Programming
See All in Programming
LT 2025-06-30: プロダクトエンジニアの役割
yamamotok
0
340
Systèmes distribués, pour le meilleur et pour le pire - BreizhCamp 2025 - Conférence
slecache
0
100
明示と暗黙 ー PHPとGoの インターフェイスの違いを知る
shimabox
2
300
都市をデータで見るってこういうこと PLATEAU属性情報入門
nokonoko1203
1
570
関数型まつりレポート for JuliaTokai #22
antimon2
0
150
関数型まつり2025登壇資料「関数プログラミングと再帰」
taisontsukada
2
850
PHPで始める振る舞い駆動開発(Behaviour-Driven Development)
ohmori_yusuke
2
180
Composerが「依存解決」のためにどんな工夫をしているか #phpcon
o0h
PRO
1
220
Team operations that are not burdened by SRE
kazatohiei
1
200
ドメインモデリングにおける抽象の役割、tagless-finalによるDSL構築、そして型安全な最適化
knih
11
2k
[初登壇@jAZUG]アプリ開発者が気になるGoogleCloud/Azure+wasm/wasi
asaringo
0
130
「ElixirでIoT!!」のこれまでとこれから
takasehideki
0
370
Featured
See All Featured
How to Think Like a Performance Engineer
csswizardry
24
1.7k
Reflections from 52 weeks, 52 projects
jeffersonlam
351
20k
The Cost Of JavaScript in 2023
addyosmani
51
8.4k
Rails Girls Zürich Keynote
gr2m
94
14k
Making Projects Easy
brettharned
116
6.3k
Build your cross-platform service in a week with App Engine
jlugia
231
18k
[RailsConf 2023] Rails as a piece of cake
palkan
55
5.6k
Product Roadmaps are Hard
iamctodd
PRO
54
11k
What’s in a name? Adding method to the madness
productmarketing
PRO
23
3.5k
Automating Front-end Workflow
addyosmani
1370
200k
ReactJS: Keep Simple. Everything can be a component!
pedronauck
667
120k
Bash Introduction
62gerente
614
210k
Transcript
A Riak Query Tale Mathias Meyer, @roidrage NoSQL Cologne
http://riakhandbook.com
Riak Distributed Database Fault-Tolerant Content-Agnostic Scalable on Demand
Querying Data
Key-Value $ curl localhost:8098/riak/users/roidrage
Links $ curl -‐v localhost:8098/riak/users/roidrage < HTTP/1.1 200 OK <
Link: </riak/users/klimpong>; riaktag="friend"
Links $ curl .../riak/users/roidrage/users,friend,_/
Listing Keys $ curl .../riak/users?keys=true
Don’t do that!
Streaming Keys $ curl .../riak/users?keys=stream
Avoid that!
Loads all the keys.
MapReduce
MapReduce Transform (Map) Aggregate (Reduce)
Warning: JavaScript
MapReduce riak.add("users"). map("Riak.mapValues").
run()
MapReduce var nameLength = function(value) { var doc
= Riak.mapValues(value)[0]; return [doc.length]; }
MapReduce riak.add("users"). map(nameLength).
run()
MapReduce riak.add("users"). map(nameLength).
reduce("Riak.reduceSum"). run()
MapReduce var average = function(values) { var avg
= values.reduce(function(n, sum) { return sum += n; }, 0); return [(avg / values.length)]; }
MapReduce riak.add("users"). map(nameLength).
reduce(average). run()
MapReduce riak.add("users"). map(nameLength).
reduce(average). run() Uh-Oh!
MapReduce riak.add(["users", "roidrage"]). map(nameLength).
reduce(average). run() Better!
JavaScript M/R Breaks with Millions of Objects Uses External Libraries
Serializes Data for JavaScript
Warning: Erlang
MapReduce riak.add('tweets'). map({language: 'erlang',
module: 'riak_kv_mapreduce', function: 'map_object_value'}).run()
MapReduce $ riak attach > {ok, C} = riak:local_client().
MapReduce C:mapred([{<<"users">>, <<"roidrage">>}], [{map, {modfun, riak_kv_mapreduce, map_object_value}, none, false}, {reduce,
{modfun, riak_kv_mapreduce, reduce_count_inputs}, none, true}]).
MapReduce ExtractFirstName1 = fun(RObject, _, _) -‐>
Value = riak_object:get_value(RObject), [FirstName, _] = re:split(Value, " "), [FirstName] end.
MapReduce C:mapred([{<<"users">>, <<"roidrage">>}],
[{map, {qfun, ExtractFirstName}, none, true}]).
Erlang M/R Much more efficient than JavaScript No serialization No
ad-hoc functions through HTTP
Key-Filters Reduce MapReduce input Based on key matches
Key-Filters riak.add({bucket: 'users', key_filters: [["matches", "^roid"]]})
Key-Filters riak.add({bucket: 'users', key_filters: [["to_upper"],
["matches", "^ROID"]]})
Key-Filters riak.add({bucket: 'users', key_filters: [["to_upper"],
["to_lower"], ["matches", "^roid"]]})
Key-Filters riak.add({bucket: 'users', key_filters: [["to_upper"],
["ends_with", "RAGE"]]})
Key-Filters riak.add({bucket: 'users', key_filters:
[["and", [["string_to_int"], ["less_than", 10]], [["string_to_int"], ["greater_than", 5]]]]})
Don't use key filters.
Riak 2i Sorted Secondary Indexes Simple Reverse Lookups Maintained Manually
Requires LevelDB
Riak 2i curl -‐X PUT .../riak/users/roidrage -‐d @-‐ \
-‐H "Content-‐Type: text/plain" \ -‐H "X-‐Riak-‐Index-‐firstname_bin: mathias" \ -‐H "X-‐Riak-‐Index-‐lastname_bin: meyer"
Riak 2i X-‐Riak-‐Index-‐firstname_bin: Mathias X-‐Riak-‐Index-‐lastname_bin: Meyer
Riak 2i X-‐Riak-‐Index-‐firstname_bin: Mathias X-‐Riak-‐Index-‐lastname_bin: Meyer X-‐Riak-‐Index-‐age_int: 34
Riak 2i X-‐Riak-‐Index-‐firstname_bin: Mathias X-‐Riak-‐Index-‐lastname_bin: Meyer X-‐Riak-‐Index-‐age_int: 34 X-‐Riak-‐Index-‐topics_bin: nosql,cloud,operations
Riak 2i # Match $ curl .../buckets/users/index/firstname_bin/Mathias
Riak 2i # Range $ curl .../buckets/users/index/firstname_bin/Mathias/Till
Riak 2i # Key $ curl .../buckets/users/index/$key/roidrage
Ordered Keys! (sort of)
MapReduce riak.add({bucket: 'users',
index: 'lastname_bin', key: 'mathias'}). map('Riak.mapValuesJson').run()
Riak 2i No Multi-Index Queries Requires Extra Work in the
App Returns only keys Document-partitioned
Riak Search Full-Text Search Solr-ish Interface Integrates with Riak
Riak Search curl -‐X PUT localhost:8098/riak/users -‐d @-‐ \
-‐H "Content-‐Type: application/json" {"props":{"precommit": [{"mod":"riak_search_kv_hook","fun":"precommit"} ]}}
Indexing Riak Objects curl -‐X PUT .../riak/users/roidrage \
-‐d "Mathias Meyer" -‐H "Content-‐Type: text/plain"
Solr-ish Interface curl .../solr/users/select?q=value:Mathias
Riak Search value:Mathias OR value:Till value:Mathias AND value:Meyer value:Mat* value:[Mathias
TO Till]
MapReduce riak.addSearch("users", "value:Mathias"). map("Riak.mapValues").run()
Riak Search Full text search of structured data Term-partitioned Efficient
for one term queries Multiple Interfaces No Anti-Entropy
When?
Key Listings Never! Almost
MapReduce Analytical Queries Fixed Dataset
Key Filters Never!
Riak 2i Simple Lookups and Range Queries Unbounded Queries Full
Fault-Tolerance
Riak Search Larger documents Full indexing Flexible queries Low frequency
terms
Questions?