MongoDB London 2011 - MongoDB Command Line Tools

Transcript

1. mongo*
   Just what do those commands do?
   By Russell Smith
   Friday, 20 April 12
   

   View Slide

2. /usr/bin/whoami
   • Russell Smith
   • Consultant for UKD1 Limited
   • I Specialise in helping companies going through rapid growth;
   • Code, architecture, infrastructure, devops, sysops, capacity planning, etc
   • <3 MongoDB, Neo4j, MySQL, Riak, Gearman, Kohana, PHP, Debian, Puppet, etc...
   Friday, 20 April 12
   

   View Slide

3. Huh?
   • MongoDB ships with a variety of CLI tools
   • Some popular;
   mongodump, mongorestore, mongostat, mongo
   • Some less so;
   mongosniff, mongoexport, mongoimport, bsondump, etc
   Friday, 20 April 12
   

   View Slide

4. Core
   Friday, 20 April 12
   

   View Slide

5. mongo
   • Probably the most commonly used
   • ‘Hidden’ gems;
   • ~/.mongorc.js - will be executed by default
   • --eval “some js” - evaluates Javascript and exits
   Friday, 20 April 12
   

   View Slide

6. mongo (cont.)
   ~ mongo test --eval "printjson(db.getCollectionNames())"
   MongoDB shell version: 2.1.0-pre-
   connecting to: test
   [ "fs.chunks", "fs.files", "system.indexes", "system.profile", "test" ]
   ~ mongo test --quiet --eval "printjson(db.getCollectionNames())"
   [ "fs.chunks", "fs.files", "system.indexes", "system.profile", "test" ]
   Friday, 20 April 12
   

   View Slide

7. ~/.mongorc.js
   x = db.queue.count({state:4});
   if (x > 0)
   {
   print('WARNING: ' + x + ' hung jobs');
   }
   Friday, 20 April 12
   

   View Slide

8. ~/.mongorc.js
   ~ mongo
   MongoDB shell version: 2.1.0-pre-
   connecting to: test
   WARNING: 2 hung jobs
   >
   Friday, 20 April 12
   

   View Slide

9. mongod
   • The server itself
   • Useful:
   • --fastsync - allows (much) faster starting of a replset member by
   using a (recent) snapshot of a peer
   • --keyFile - replset / sharding authentication file
   Friday, 20 April 12
   

   View Slide

10. mongos
    • Sits in front of your shard servers and directs queries
    • --keyFile
    Friday, 20 April 12
    

    View Slide

11. Common
    Friday, 20 April 12
    

    View Slide

12. mongodump
    • Used to take backups
    • Common options:
    • --oplog
    • -d
    • -c
    Friday, 20 April 12
    

    View Slide

13. mongodump (cont.)
    ~ mongodump -d test -c queue -q "{state:4}"
    connected to: 127.0.0.1
    DATABASE: test
    to
    dump/test
    
    test.queue to dump/test/queue.bson
    
    
    2 objects
    Friday, 20 April 12
    

    View Slide

14. mongorestore
    • Restores backups taken with mongodump
    • Interesting options;
    • --filter
    • --oplogReplay
    • Possible performance issues with custom (non ObjectId) _id
    Friday, 20 April 12
    

    View Slide

15. mongorestore (cont.)
    mongorestore -d test -c queue --filter "{state:4}" dump/test/queue.bson
    connected to: 127.0.0.1
    Mon Sep 19 07:34:03 dump/test/queue.bson
    Mon Sep 19 07:34:03
    going into namespace [test.queue]
    764 objects found
    2 objects processed
    Friday, 20 April 12
    

    View Slide

16. mongostat
    • Produces periodically updated stats
    • Useful in watching server load & spotting general performance issues
    Friday, 20 April 12
    

    View Slide

17. Cursor
    paging
    Common
    operations
    fsync
    flushes
    Total
    data size
    locked
    active
    clients
    queues
    Friday, 20 April 12
    

    View Slide

18. mongostat (cont’d)
    • Watch for;
    • faults (page faults)
    • idx miss (index btree page misses)
    • locked (percentage of time globally locked)
    • repl (it should normally be M or SEC)
    Friday, 20 April 12
    

    View Slide

19. Others
    Friday, 20 April 12
    

    View Slide

20. mongoexport
    • Useful to export data for use with another database or program
    • You may specify down to the field level using a filter if needed
    • Can export as;
    • JSON (default)
    • CSV
    Friday, 20 April 12
    

    View Slide

21. mongoimport
    • Imports data in to mongodb
    • Currently supports importing;
    • JSON
    • CSV
    • TSV
    Friday, 20 April 12
    

    View Slide

22. mongoimport (cont’d)
    • Useful options include;
    • --upsert - allows updating objects that exist already
    (see --upsertFields)
    • --headerline - if your CSV / TSV contains headers
    • -f / --fields - specify field names
    Friday, 20 April 12
    

    View Slide

23. bsondump
    • Convert bson -> json
    bsondump test.bson --type json
    { "_id" :
    ObjectId( "4e72f51414345f302b0eea2e" ),
    "test" : 1 }
    1 objects found
    Friday, 20 April 12
    

    View Slide

24. bsondump (cont.)
    • Debug info:
    bsondump test.bson --type debug
    --- new object ---
    
    size : 36
    
    
    _id
    
    
    
    type: 7 size: 17
    
    
    test
    
    
    
    type: 1 size: 14
    etc
    Friday, 20 April 12
    

    View Slide

25. mongofiles
    ~ mongofiles put test.txt
    connected to: 127.0.0.1
    added file: { _id: ObjectId('4e761b07c21062d06b349819'), filename:
    "test.txt", chunkSize: 262144, uploadDate: new Date(1316363015625),
    md5: "d41d8cd98f00b204e9800998ecf8427e", length: 12256 }
    done!
    Friday, 20 April 12
    

    View Slide

26. mongofiles (cont.)
    ~ mongofiles list
    connected to: 127.0.0.1
    test.txt
    12256
    Friday, 20 April 12
    

    View Slide

27. mongotop
    • New in 1.9
    • Collection level load stats
    Friday, 20 April 12
    

    View Slide

28. mongosniff
    • Used for sniffing the traffic between client / server
    • Can forward traffic
    • Only really useful for debugging...
    Friday, 20 April 12
    

    View Slide

29. Friday, 20 April 12
    

    View Slide

30. mongo-perf
    • Does not ship with mongodb
    See : https://github.com/mongodb/mongo-perf
    • I can’t compile it, but it probably works
    Friday, 20 April 12
    

    View Slide

31. Useful third party tools
    Friday, 20 April 12
    

    View Slide

32. tungsten replicator
    • http://code.google.com/p/tungsten-replicator/
    • http://code.google.com/p/tungsten-replicator/wiki/
    mysql> create table foo(id int primary key, msg varchar(35));
    Query OK, 0 rows affected (0.05 sec)
    mysql> insert into foo values(1, 'hello from MySQL!');
    Query OK, 1 row affected (0.00 sec)
    > db.foo.find()
    { "_id" : ObjectId("4dc77bacad9092bd1aef046d"), "id" : "1", "data" :
    "hello from MySQL!" }
    Friday, 20 April 12
    

    View Slide

33. automongobackup
    • A port of the popular AutoMySQLBackup
    • Rotates & compresses backups
    • https://github.com/micahwedemeyer/automongobackup
    Friday, 20 April 12
    

    View Slide

34. mongolog
    • Full query logging
    • Uses mongosniff
    • Will be on github soon
    Friday, 20 April 12
    

    View Slide

35. Other interesting projects
    • Eliots Munin plugins : https://github.com/erh/mongo-munin
    • gridfs-fuse : https://github.com/mikejs/gridfs-fuse
    Friday, 20 April 12
    

    View Slide

36. Further reading
    • Tungsten with Mongo details
    http://scale-out-blog.blogspot.com/2011/05/introducing-mysql-to-mongodb.html
    • Javascript shell API
    http://api.mongodb.org/js/current/
    • Admin UI’s
    http://www.mongodb.org/display/DOCS/Admin+UIs
    • Hacking the shell
    http://bit.ly/r2vghH
    Friday, 20 April 12
    

    View Slide

37. Questions?
    Friday, 20 April 12
    

    View Slide