KEEP YOUR DATA IN SYNC IN YOUR ERLANG APPLICATION CodeBeam Munich - 2018 

benoît chesneau craftsman working on P2P and custom data endpoints solution Created 11 years ago, Enki Multimedia member of the Erlang Industrial User Group about me 

more and more of our data is in the "Cloud" we let to third parties the control over our data we live in the time where Making us more and more dependant 

Barrel is not a database. It is a framework / product that synchronizes 
 and replicates data between machines and applications. 

keep a partial state of your data locally replicate changes in your data route automatically or implicitly a view of your data Built in Erlang Opensource Features 

Barrel, core concepts 

core data structure of barrel are documents stored in a barrel can be represented as a an Erlang map or JSON with a revision Document oriented { "id" : "ID", "key" : "value", "_rev": "REVISIONID" } 

attachments are binary contents updated or linked (http) to a document attach anything { "id" : "ID", "key" : "value", "_attachments": [ { "id" : "AttachmentID" } ] } 

by document IDs by path automatically indexed { "id" : "ID", "locations" : [ { "country" : "France", "city" : "creil" } ], "headquarter" : "France" } $/id/ID $/locations/0/country/france $/locations/0/city/creil $/headquarter/france 

listen on any document changes changes are incremental filter changes persist view changes database changes { "id" : "ID", "rev" : "REVID", "seq" : 1, "changes" : ["REVID"] } 

uses the changes feed handle the conflicts push/pull replication push/pull replication 

built in Erlang good resilience due to an efficient supervision. (aten from rabbitmq for local nodes monitoring) Each steps of the write workflow is done in different isolated processes local document db using Rocksdb using dirty-nifs:
 https://gitlab.com/barrel-db/erlang-rocksdb new improvements in the TCP/IO stack improves the performance for free. 

create a custom data platform mixing local and remote storages and different peer-to-peer replications strategies. 

HTTP(s) & Websocket for any one lib can be embedded in an Erlang Node if you want to remote nodes uses websockets to exchange the data embedded or remote 

barrels have to be created per nodes Or via the HTTP API create a barrel Name = <<"mybarrel">>, Params = #{ store_provider => <<"MyLocalRocksdb">> }, ok = create_barrel(Name, Params) POST /barrels/name { "name": "mybarrel", "params" : { "store_provider": "MyLocalRocksdb" } } 

once opened , you can use a barrel to create and retrieve a doc simple put and get a barrel {ok, Barrel} = barrel:open(Name) {ok, DocId, Rev} = barrel:save_doc(Barrel, {}), {ok, Doc = #{ <<"id">> := DocId, <<"_rev">> := Rev }} = barrel:fetch_doc(Barrel, DocId) 

once opened , you can use a barrel to create and retrieve a doc can be nested "Peer to Peer" Put Params = #{ p2p_write => { host => "HOST:PORT", name => <<"RemoteBarrel">> }} {ok, DocId, Rev} = barrel:save_doc(Barrel, {}, Params) Params = #{p2p_write => { host => "HOST:PORT", name => <<"RemoteBarrel">>, p2p_write => { ... } }} 

similar syntax to Put Read first locally then remote "Peer to Peer" Get Params = #{p2p_read => { host => "HOST:PORT", name => <<"RemoteBarrel">>} {ok, Doc} = barrel:fetch_doc(Barrel, {}, Params), 

once opened , you can use a barrel to create and retrieve a doc
 
 Source and Target can be remote or local. Or both. Can work in Push and Pull mode at the same time (synchronisation). REPLICATION Params = #{ <<"source">> => <<"MyLocalBarrel">>, <<"target" => << "ws://HOST:PORT/RemoteBarrel" >> } {ok, RepId} = barrel:replicate(Params), 

A listener is a simple process on which you can pass a callback. you can subscribe to different pattern of path with a semantic similar to the one in MQTT Listen on change Callback = fun(Change) -> ok end, Params => #{ callback_fun => MyCallback, subscribe_to => [<<"/*">>], since => 0 }, {ok, ListenerPid} = start_listener(Barrel, Params). 

what's next? 

[email protected] https://barrel-db.org