Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Tools, Testing and Disasters at Basho

Tom Santero
October 18, 2012

Tools, Testing and Disasters at Basho

Slides from my talk at the DevOpsATL meetup on 10/18/2012. [http://www.meetup.com/DevOpsATL/events/82032472/] Discussing Riak, DevOps at Basho and what happens when things go wrong.

Tom Santero

October 18, 2012
Tweet

More Decks by Tom Santero

Other Decks in Technology

Transcript

  1. Tom Santero @tsantero October 18, 2012 - ATL DevOps Meetup

    Tools, Testing and Disasters Basho at Friday, October 19, 12
  2. $ whoami Tom Santero @tsantero on Twitter [email protected] I <3

    open source software, beer and kitty cats github.com/tsantero Friday, October 19, 12
  3. key/value store key/value pairs mapped to buckets values are opaque

    objects stored as binaries on disk key value key value key value key value bucket Friday, October 19, 12
  4. masterless a deployed as a cluster of nodes e b

    d c any node can serve any request Friday, October 19, 12
  5. masterless a deployed as a cluster of nodes request e

    b d c any node can serve any request Friday, October 19, 12
  6. masterless a deployed as a cluster of nodes e b

    d c any node can serve any request Friday, October 19, 12
  7. masterless a deployed as a cluster of nodes e b

    d c any node can serve any request payload Friday, October 19, 12
  8. consistent hashing node 0 node 1 node 2 node 3

    hash(“meetups/DevOpsATL”) N = 3 Friday, October 19, 12
  9. consistent hashing node 0 node 1 node 2 node 3

    hash(“meetups/DevOpsATL”) N = 3 Friday, October 19, 12
  10. disaster scenario node 0 node 1 node 3 node 2

    requests go to fallback Friday, October 19, 12
  11. disaster scenario node 0 node 1 node 3 node 2

    node comes back online Friday, October 19, 12
  12. disaster scenario node 0 node 1 node 3 node 2

    normal operations resume Friday, October 19, 12
  13. access your programmatically DATA with Erlang Ruby Python Java Python

    Haskell Perl Go Clojure OCaml Node.JS Dart ...your language of choice Friday, October 19, 12
  14. GET PUT DELETE CRUD CRUD CRUD CRUD CRUD CRUD CRUD

    CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD CRUD Friday, October 19, 12
  15. store and index documents Riak Search JSON / XML plain

    text english prose query against VALUES Friday, October 19, 12
  16. 2i Secondary Indexes Riak Object Riak Object Riak Object tag

    objects X-Riak-Index-email_bin X-Riak-Index-email_bin X-Riak-Index-email_bin “[email protected]” “[email protected]” “[email protected]” “[email protected]” Friday, October 19, 12
  17. Ask 10 engineers what they think DevOps is and you’ll

    get 12.79234 di"erent answers Friday, October 19, 12
  18. guttersnipe middleware in Vagrant deploy and run test suite no

    need to rebuild clusters 500% increase in e"ciency Friday, October 19, 12
  19. https://github.com/basho/riak-chef-cookbook CHEF COOKBOOKS use to deploy riak open source over

    40 commits in 2012 rolling upgrades, iptables, etc... future development Friday, October 19, 12
  20. PUPPET MODULES https://github.com/basho/puppet-riak use to deploy riak just open sourced

    I don’t know puppet, so I’m not going to bullshit you Friday, October 19, 12
  21. PUPPET MODULES https://github.com/basho/puppet-riak use to deploy riak just open sourced

    I don’t know puppet, so I’m not going to bullshit you Friday, October 19, 12
  22. RIAK TEST https://github.com/basho/riak_test utility for testing builds run tests in

    a sandbox reset to clean state after tests easy to bootstrap envs also open source! Friday, October 19, 12
  23. GiddyUp https://github.com/basho/giddyup visual scorecard for Riak Test seed Riak Test

    with list of tests to be run on each platform receive test results and logs via REST interface Friday, October 19, 12
  24. DataCenter 300 cpu cores 1 TB RAM 500 TB storage

    gigabit internet Friday, October 19, 12
  25. DataCenter 300 cpu cores 1 TB RAM 500 TB storage

    gigabit internet Friday, October 19, 12
  26. “In both cases I was unable to replicate these issues

    on the hardware available to me locally, both through limited machines (2) and major performance di!erences between nodes ("rst gen mbp vs i7 dell tower)” --Andrew Thompson, Basho Sr. Engineer Friday, October 19, 12
  27. !n hugs! Any and all questions can be sent to

    /dev/null Friday, October 19, 12