MongoDB London 2011 - MongoDB Command Line Tools

Transcript

1. mongo* Just what do those commands do? By Russell Smith

   Friday, 20 April 12

2. /usr/bin/whoami • Russell Smith • Consultant for UKD1 Limited •

   I Specialise in helping companies going through rapid growth; • Code, architecture, infrastructure, devops, sysops, capacity planning, etc • <3 MongoDB, Neo4j, MySQL, Riak, Gearman, Kohana, PHP, Debian, Puppet, etc... Friday, 20 April 12

3. Huh? • MongoDB ships with a variety of CLI tools

   • Some popular; mongodump, mongorestore, mongostat, mongo • Some less so; mongosniff, mongoexport, mongoimport, bsondump, etc Friday, 20 April 12

4. Core Friday, 20 April 12

5. mongo • Probably the most commonly used • ‘Hidden’ gems;

   • ~/.mongorc.js - will be executed by default • --eval “some js” - evaluates Javascript and exits Friday, 20 April 12

6. mongo (cont.) ~ mongo test --eval "printjson(db.getCollectionNames())" MongoDB shell version:

   2.1.0-pre- connecting to: test [ "fs.chunks", "fs.files", "system.indexes", "system.profile", "test" ] ~ mongo test --quiet --eval "printjson(db.getCollectionNames())" [ "fs.chunks", "fs.files", "system.indexes", "system.profile", "test" ] Friday, 20 April 12

7. ~/.mongorc.js x = db.queue.count({state:4}); if (x > 0) { print('WARNING:

   ' + x + ' hung jobs'); } Friday, 20 April 12

8. ~/.mongorc.js ~ mongo MongoDB shell version: 2.1.0-pre- connecting to: test

   WARNING: 2 hung jobs > Friday, 20 April 12

9. mongod • The server itself • Useful: • --fastsync -

   allows (much) faster starting of a replset member by using a (recent) snapshot of a peer • --keyFile - replset / sharding authentication file Friday, 20 April 12

10. mongos • Sits in front of your shard servers and

    directs queries • --keyFile Friday, 20 April 12

11. Common Friday, 20 April 12

12. mongodump • Used to take backups • Common options: •

    --oplog • -d • -c Friday, 20 April 12

13. mongodump (cont.) ~ mongodump -d test -c queue -q "{state:4}"

    connected to: 127.0.0.1 DATABASE: test to dump/test test.queue to dump/test/queue.bson 2 objects Friday, 20 April 12

14. mongorestore • Restores backups taken with mongodump • Interesting options;

    • --filter • --oplogReplay • Possible performance issues with custom (non ObjectId) _id Friday, 20 April 12

15. mongorestore (cont.) mongorestore -d test -c queue --filter "{state:4}" dump/test/queue.bson

    connected to: 127.0.0.1 Mon Sep 19 07:34:03 dump/test/queue.bson Mon Sep 19 07:34:03 going into namespace [test.queue] 764 objects found 2 objects processed Friday, 20 April 12

16. mongostat • Produces periodically updated stats • Useful in watching

    server load & spotting general performance issues Friday, 20 April 12

17. Cursor paging Common operations fsync flushes Total data size locked

    active clients queues Friday, 20 April 12

18. mongostat (cont’d) • Watch for; • faults (page faults) •

    idx miss (index btree page misses) • locked (percentage of time globally locked) • repl (it should normally be M or SEC) Friday, 20 April 12

19. Others Friday, 20 April 12

20. mongoexport • Useful to export data for use with another

    database or program • You may specify down to the field level using a filter if needed • Can export as; • JSON (default) • CSV Friday, 20 April 12

21. mongoimport • Imports data in to mongodb • Currently supports

    importing; • JSON • CSV • TSV Friday, 20 April 12

22. mongoimport (cont’d) • Useful options include; • --upsert - allows

    updating objects that exist already (see --upsertFields) • --headerline - if your CSV / TSV contains headers • -f / --fields - specify field names Friday, 20 April 12

23. bsondump • Convert bson -> json bsondump test.bson --type json

    { "_id" : ObjectId( "4e72f51414345f302b0eea2e" ), "test" : 1 } 1 objects found Friday, 20 April 12

24. bsondump (cont.) • Debug info: bsondump test.bson --type debug ---

    new object --- size : 36 _id type: 7 size: 17 test type: 1 size: 14 etc Friday, 20 April 12

25. mongofiles ~ mongofiles put test.txt connected to: 127.0.0.1 added file:

    { _id: ObjectId('4e761b07c21062d06b349819'), filename: "test.txt", chunkSize: 262144, uploadDate: new Date(1316363015625), md5: "d41d8cd98f00b204e9800998ecf8427e", length: 12256 } done! Friday, 20 April 12

26. mongofiles (cont.) ~ mongofiles list connected to: 127.0.0.1 test.txt 12256

    Friday, 20 April 12

27. mongotop • New in 1.9 • Collection level load stats

    Friday, 20 April 12

28. mongosniff • Used for sniffing the traffic between client /

    server • Can forward traffic • Only really useful for debugging... Friday, 20 April 12

29. Friday, 20 April 12

30. mongo-perf • Does not ship with mongodb See : https://github.com/mongodb/mongo-perf

    • I can’t compile it, but it probably works Friday, 20 April 12

31. Useful third party tools Friday, 20 April 12

32. tungsten replicator • http://code.google.com/p/tungsten-replicator/ • http://code.google.com/p/tungsten-replicator/wiki/ mysql> create table foo(id

    int primary key, msg varchar(35)); Query OK, 0 rows affected (0.05 sec) mysql> insert into foo values(1, 'hello from MySQL!'); Query OK, 1 row affected (0.00 sec) > db.foo.find() { "_id" : ObjectId("4dc77bacad9092bd1aef046d"), "id" : "1", "data" : "hello from MySQL!" } Friday, 20 April 12

33. automongobackup • A port of the popular AutoMySQLBackup • Rotates

    & compresses backups • https://github.com/micahwedemeyer/automongobackup Friday, 20 April 12

34. mongolog • Full query logging • Uses mongosniff • Will

    be on github soon Friday, 20 April 12

35. Other interesting projects • Eliots Munin plugins : https://github.com/erh/mongo-munin •

    gridfs-fuse : https://github.com/mikejs/gridfs-fuse Friday, 20 April 12

36. Further reading • Tungsten with Mongo details http://scale-out-blog.blogspot.com/2011/05/introducing-mysql-to-mongodb.html • Javascript

    shell API http://api.mongodb.org/js/current/ • Admin UI’s http://www.mongodb.org/display/DOCS/Admin+UIs • Hacking the shell http://bit.ly/r2vghH Friday, 20 April 12

37. Questions? Friday, 20 April 12