Upgrade to Pro — share decks privately, control downloads, hide ads and more …

MongoDB London 2011 - MongoDB Command Line Tools

MongoDB London 2011 - MongoDB Command Line Tools

Russell Smith

April 20, 2012
Tweet

More Decks by Russell Smith

Other Decks in Technology

Transcript

  1. mongo*
    Just what do those commands do?
    By Russell Smith
    Friday, 20 April 12

    View Slide

  2. /usr/bin/whoami
    • Russell Smith
    • Consultant for UKD1 Limited
    • I Specialise in helping companies going through rapid growth;
    • Code, architecture, infrastructure, devops, sysops, capacity planning, etc
    • <3 MongoDB, Neo4j, MySQL, Riak, Gearman, Kohana, PHP, Debian, Puppet, etc...
    Friday, 20 April 12

    View Slide

  3. Huh?
    • MongoDB ships with a variety of CLI tools
    • Some popular;
    mongodump, mongorestore, mongostat, mongo
    • Some less so;
    mongosniff, mongoexport, mongoimport, bsondump, etc
    Friday, 20 April 12

    View Slide

  4. Core
    Friday, 20 April 12

    View Slide

  5. mongo
    • Probably the most commonly used
    • ‘Hidden’ gems;
    • ~/.mongorc.js - will be executed by default
    • --eval “some js” - evaluates Javascript and exits
    Friday, 20 April 12

    View Slide

  6. mongo (cont.)
    ~ mongo test --eval "printjson(db.getCollectionNames())"
    MongoDB shell version: 2.1.0-pre-
    connecting to: test
    [ "fs.chunks", "fs.files", "system.indexes", "system.profile", "test" ]
    ~ mongo test --quiet --eval "printjson(db.getCollectionNames())"
    [ "fs.chunks", "fs.files", "system.indexes", "system.profile", "test" ]
    Friday, 20 April 12

    View Slide

  7. ~/.mongorc.js
    x = db.queue.count({state:4});
    if (x > 0)
    {
    print('WARNING: ' + x + ' hung jobs');
    }
    Friday, 20 April 12

    View Slide

  8. ~/.mongorc.js
    ~ mongo
    MongoDB shell version: 2.1.0-pre-
    connecting to: test
    WARNING: 2 hung jobs
    >
    Friday, 20 April 12

    View Slide

  9. mongod
    • The server itself
    • Useful:
    • --fastsync - allows (much) faster starting of a replset member by
    using a (recent) snapshot of a peer
    • --keyFile - replset / sharding authentication file
    Friday, 20 April 12

    View Slide

  10. mongos
    • Sits in front of your shard servers and directs queries
    • --keyFile
    Friday, 20 April 12

    View Slide

  11. Common
    Friday, 20 April 12

    View Slide

  12. mongodump
    • Used to take backups
    • Common options:
    • --oplog
    • -d
    • -c
    Friday, 20 April 12

    View Slide

  13. mongodump (cont.)
    ~ mongodump -d test -c queue -q "{state:4}"
    connected to: 127.0.0.1
    DATABASE: test
    to
    dump/test

    test.queue to dump/test/queue.bson


    2 objects
    Friday, 20 April 12

    View Slide

  14. mongorestore
    • Restores backups taken with mongodump
    • Interesting options;
    • --filter
    • --oplogReplay
    • Possible performance issues with custom (non ObjectId) _id
    Friday, 20 April 12

    View Slide

  15. mongorestore (cont.)
    mongorestore -d test -c queue --filter "{state:4}" dump/test/queue.bson
    connected to: 127.0.0.1
    Mon Sep 19 07:34:03 dump/test/queue.bson
    Mon Sep 19 07:34:03
    going into namespace [test.queue]
    764 objects found
    2 objects processed
    Friday, 20 April 12

    View Slide

  16. mongostat
    • Produces periodically updated stats
    • Useful in watching server load & spotting general performance issues
    Friday, 20 April 12

    View Slide

  17. Cursor
    paging
    Common
    operations
    fsync
    flushes
    Total
    data size
    locked
    active
    clients
    queues
    Friday, 20 April 12

    View Slide

  18. mongostat (cont’d)
    • Watch for;
    • faults (page faults)
    • idx miss (index btree page misses)
    • locked (percentage of time globally locked)
    • repl (it should normally be M or SEC)
    Friday, 20 April 12

    View Slide

  19. Others
    Friday, 20 April 12

    View Slide

  20. mongoexport
    • Useful to export data for use with another database or program
    • You may specify down to the field level using a filter if needed
    • Can export as;
    • JSON (default)
    • CSV
    Friday, 20 April 12

    View Slide

  21. mongoimport
    • Imports data in to mongodb
    • Currently supports importing;
    • JSON
    • CSV
    • TSV
    Friday, 20 April 12

    View Slide

  22. mongoimport (cont’d)
    • Useful options include;
    • --upsert - allows updating objects that exist already
    (see --upsertFields)
    • --headerline - if your CSV / TSV contains headers
    • -f / --fields - specify field names
    Friday, 20 April 12

    View Slide

  23. bsondump
    • Convert bson -> json
    bsondump test.bson --type json
    { "_id" :
    ObjectId( "4e72f51414345f302b0eea2e" ),
    "test" : 1 }
    1 objects found
    Friday, 20 April 12

    View Slide

  24. bsondump (cont.)
    • Debug info:
    bsondump test.bson --type debug
    --- new object ---

    size : 36


    _id



    type: 7 size: 17


    test



    type: 1 size: 14
    etc
    Friday, 20 April 12

    View Slide

  25. mongofiles
    ~ mongofiles put test.txt
    connected to: 127.0.0.1
    added file: { _id: ObjectId('4e761b07c21062d06b349819'), filename:
    "test.txt", chunkSize: 262144, uploadDate: new Date(1316363015625),
    md5: "d41d8cd98f00b204e9800998ecf8427e", length: 12256 }
    done!
    Friday, 20 April 12

    View Slide

  26. mongofiles (cont.)
    ~ mongofiles list
    connected to: 127.0.0.1
    test.txt
    12256
    Friday, 20 April 12

    View Slide

  27. mongotop
    • New in 1.9
    • Collection level load stats
    Friday, 20 April 12

    View Slide

  28. mongosniff
    • Used for sniffing the traffic between client / server
    • Can forward traffic
    • Only really useful for debugging...
    Friday, 20 April 12

    View Slide

  29. Friday, 20 April 12

    View Slide

  30. mongo-perf
    • Does not ship with mongodb
    See : https://github.com/mongodb/mongo-perf
    • I can’t compile it, but it probably works
    Friday, 20 April 12

    View Slide

  31. Useful third party tools
    Friday, 20 April 12

    View Slide

  32. tungsten replicator
    • http://code.google.com/p/tungsten-replicator/
    • http://code.google.com/p/tungsten-replicator/wiki/
    mysql> create table foo(id int primary key, msg varchar(35));
    Query OK, 0 rows affected (0.05 sec)
    mysql> insert into foo values(1, 'hello from MySQL!');
    Query OK, 1 row affected (0.00 sec)
    > db.foo.find()
    { "_id" : ObjectId("4dc77bacad9092bd1aef046d"), "id" : "1", "data" :
    "hello from MySQL!" }
    Friday, 20 April 12

    View Slide

  33. automongobackup
    • A port of the popular AutoMySQLBackup
    • Rotates & compresses backups
    • https://github.com/micahwedemeyer/automongobackup
    Friday, 20 April 12

    View Slide

  34. mongolog
    • Full query logging
    • Uses mongosniff
    • Will be on github soon
    Friday, 20 April 12

    View Slide

  35. Other interesting projects
    • Eliots Munin plugins : https://github.com/erh/mongo-munin
    • gridfs-fuse : https://github.com/mikejs/gridfs-fuse
    Friday, 20 April 12

    View Slide

  36. Further reading
    • Tungsten with Mongo details
    http://scale-out-blog.blogspot.com/2011/05/introducing-mysql-to-mongodb.html
    • Javascript shell API
    http://api.mongodb.org/js/current/
    • Admin UI’s
    http://www.mongodb.org/display/DOCS/Admin+UIs
    • Hacking the shell
    http://bit.ly/r2vghH
    Friday, 20 April 12

    View Slide

  37. Questions?
    Friday, 20 April 12

    View Slide