backup and restore
- journaling must be enabled on the logical volume.
To backup, start by using command mongodump.
jason@localhost:~$ mongodump
connected to: 127.0.0.1
2014-07-07T22:46:09.351+0800 all dbs
2014-07-07T22:46:09.352+0800 DATABASE: test to dump/test
2014-07-07T22:46:09.354+0800 test.system.indexes to dump/test/system.indexes.bson
2014-07-07T22:46:09.355+0800 4 documents
2014-07-07T22:46:09.356+0800 test.testData to dump/test/testData.bson
2014-07-07T22:46:09.359+0800 400 documents
2014-07-07T22:46:09.360+0800 Metadata for test.testData to dump/test/testData.metadata.json
2014-07-07T22:46:09.360+0800 test.users to dump/test/users.bson
2014-07-07T22:46:09.361+0800 1 documents
2014-07-07T22:46:09.362+0800 Metadata for test.users to dump/test/users.metadata.json
2014-07-07T22:46:09.362+0800 test.accounts to dump/test/accounts.bson
2014-07-07T22:46:09.369+0800 2 documents
2014-07-07T22:46:09.370+0800 Metadata for test.accounts to dump/test/accounts.metadata.json
2014-07-07T22:46:09.370+0800 test.transactions to dump/test/transactions.bson
2014-07-07T22:46:09.372+0800 1 documents
2014-07-07T22:46:09.373+0800 Metadata for test.transactions to dump/test/transactions.metadata.json
2014-07-07T22:46:09.374+0800 DATABASE: mydb to dump/mydb
2014-07-07T22:46:09.375+0800 mydb.system.indexes to dump/mydb/system.indexes.bson
2014-07-07T22:46:09.376+0800 2 documents
2014-07-07T22:46:09.377+0800 mydb.testData to dump/mydb/testData.bson
2014-07-07T22:46:09.378+0800 27 documents
2014-07-07T22:46:09.389+0800 Metadata for mydb.testData to dump/mydb/testData.metadata.json
2014-07-07T22:46:09.390+0800 mydb.users to dump/mydb/users.bson
2014-07-07T22:46:09.391+0800 1 documents
2014-07-07T22:46:09.392+0800 Metadata for mydb.users to dump/mydb/users.metadata.json
2014-07-07T22:46:09.392+0800 DATABASE: mp3db to dump/mp3db
2014-07-07T22:46:09.393+0800 mp3db.system.indexes to dump/mp3db/system.indexes.bson
2014-07-07T22:46:09.394+0800 4 documents
2014-07-07T22:46:09.395+0800 mp3db.mp3.files to dump/mp3db/mp3.files.bson
2014-07-07T22:46:09.396+0800 1 documents
2014-07-07T22:46:09.397+0800 Metadata for mp3db.mp3.files to dump/mp3db/mp3.files.metadata.json
2014-07-07T22:46:09.397+0800 mp3db.mp3.chunks to dump/mp3db/mp3.chunks.bson
2014-07-07T22:46:09.401+0800 2 documents
2014-07-07T22:46:09.401+0800 Metadata for mp3db.mp3.chunks to dump/mp3db/mp3.chunks.metadata.json
2014-07-07T22:46:09.402+0800 DATABASE: admin to dump/admin
2014-07-07T22:46:09.406+0800 DATABASE: config to dump/config
Let's remove some data from the database before we restore.
jason@localhost:~$ mongo
MongoDB shell version: 2.6.3
connecting to: test
Server has startup warnings:
2014-06-24T21:23:40.227+0800 [initandlisten]
2014-06-24T21:23:40.227+0800 [initandlisten] ** NOTE: This is a 32 bit MongoDB binary.
2014-06-24T21:23:40.227+0800 [initandlisten] ** 32 bit builds are limited to less than 2GB of data (or less with --journal).
2014-06-24T21:23:40.227+0800 [initandlisten] ** Note that journaling defaults to off for 32 bit and is currently off.
2014-06-24T21:23:40.228+0800 [initandlisten] ** See http://dochub.mongodb.org/core/32bit
2014-06-24T21:23:40.228+0800 [initandlisten]
> use mp3db;
switched to db mp3db
> show tables;
mp3.chunks
mp3.files
system.indexes
> db.mp3.chunks.remove({});
WriteResult({ "nRemoved" : 2 })
> db.mp3.files.remove({});
WriteResult({ "nRemoved" : 1 })
> db.mp3.chunks.find();
> db.mp3.files.find();
>
Now we restore using command mongorestore.
jason@localhost:~$ mongorestore --collection mp3.chunks --db mp3db dump/mp3db/mp3.chunks.bson
connected to: 127.0.0.1
2014-07-07T23:14:43.504+0800 dump/mp3db/mp3.chunks.bson
2014-07-07T23:14:43.504+0800 going into namespace [mp3db.mp3.chunks]
Restoring to mp3db.mp3.chunks without dropping. Restored data will be inserted without raising errors; check your server log
2 objects found
2014-07-07T23:14:43.534+0800 Creating index: { key: { _id: 1 }, name: "_id_", ns: "mp3db.mp3.chunks" }
2014-07-07T23:14:43.635+0800 Creating index: { key: { files_id: 1, n: 1 }, name: "files_id_1_n_1", ns: "mp3db.mp3.chunks" }
jason@localhost:~$ mongorestore --collection mp3.files --db mp3db dump/mp3db/mp3.files.bson
connected to: 127.0.0.1
2014-07-07T23:17:24.813+0800 dump/mp3db/mp3.files.bson
2014-07-07T23:17:24.813+0800 going into namespace [mp3db.mp3.files]
Restoring to mp3db.mp3.files without dropping. Restored data will be inserted without raising errors; check your server log
1 objects found
2014-07-07T23:17:24.819+0800 Creating index: { key: { _id: 1 }, name: "_id_", ns: "mp3db.mp3.files" }
2014-07-07T23:17:24.822+0800 Creating index: { key: { filename: 1, uploadDate: 1 }, name: "filename_1_uploadDate_1", ns: "mp3db.mp3.files" }
Looks good, the restoration process and now we verify the content.
jason@localhost:~$ mongo
MongoDB shell version: 2.6.3
connecting to: test
Server has startup warnings:
2014-06-24T21:23:40.227+0800 [initandlisten]
2014-06-24T21:23:40.227+0800 [initandlisten] ** NOTE: This is a 32 bit MongoDB binary.
2014-06-24T21:23:40.227+0800 [initandlisten] ** 32 bit builds are limited to less than 2GB of data (or less with --journal).
2014-06-24T21:23:40.227+0800 [initandlisten] ** Note that journaling defaults to off for 32 bit and is currently off.
2014-06-24T21:23:40.228+0800 [initandlisten] ** See http://dochub.mongodb.org/core/32bit
2014-06-24T21:23:40.228+0800 [initandlisten]
> use mp3db;
switched to db mp3db
> db.mp3.files.find();
{ "_id" : ObjectId("53ad61c844ae8a6ee12fcb63"), "chunkSize" : NumberLong(262144), "length" : NumberLong(316773), "md5" : "7293e9fd795e2bb6d5035e5b69cb2923", "filename" : "django.mp3", "contentType" : "audio/mpeg", "uploadDate" : ISODate("2014-06-27T12:21:28.646Z"), "aliases" : null }
>
Looks good. Now we move on to monitoring.
monitoring
mongostats - captures and returns the counts of database operations by type (e.g. insert, query, update, delete, etc.). These counts report on the load distribution on the server.
jason@localhost:~$ mongostat
connected to: 127.0.0.1
insert query update delete getmore command flushes mapped vsize res faults locked db idx miss % qr|qw ar|aw netIn netOut conn time
*0 *0 *0 *0 0 1|0 0 320m 445m 10m 14 config:0.0% 0 0|0 0|0 62b 3k 1 22:33:54
*0 *0 *0 *0 0 1|0 0 320m 445m 10m 0 test:0.0% 0 0|0 0|0 62b 3k 1 22:33:55
*0 *0 *0 *0 0 1|0 0 320m 445m 10m 0 test:0.0% 0 0|0 0|0 62b 3k 1 22:33:56
^C
mongotop tracks and reports the current read and write activity of a MongoDB instance, and reports these statistics on a per collection basis.
jason@localhost:~$ mongotop
connected to: 127.0.0.1
ns total read write 2014-07-07T15:20:38
mp3db.mp3.chunks 0ms 0ms 0ms
local.system.replset 0ms 0ms 0ms
local.system.namespaces 0ms 0ms 0ms
local.system.indexes 0ms 0ms 0ms
local.startup_log 0ms 0ms 0ms
config.version 0ms 0ms 0ms
config.system.namespaces 0ms 0ms 0ms
ns total read write 2014-07-07T15:20:39
mp3db.mp3.chunks 0ms 0ms 0ms
local.system.replset 0ms 0ms 0ms
local.system.namespaces 0ms 0ms 0ms
local.system.indexes 0ms 0ms 0ms
local.startup_log 0ms 0ms 0ms
config.version 0ms 0ms 0ms
config.system.namespaces 0ms 0ms 0ms
^C
jason@localhost:~$
HTTP Console - MongoDB provides a web interface that exposes diagnostic and monitoring information in a simple web page. For example , by accessing http://192.168.0.2:27017/
Now using db.serverStatus() from the mongo shell. The serverStatus command, or db.serverStatus() from the shell, returns a general overview of the status of the database, detailing disk usage, memory use, connection, journaling, and index access. The command returns quickly and does not impact MongoDB performance.
> db.serverStatus()
{
"host" : "debby.e2e.serveftp.net",
"version" : "2.6.3",
"process" : "mongod",
"pid" : NumberLong(3651),
"uptime" : 1130615,
"uptimeMillis" : NumberLong(1130614302),
"uptimeEstimate" : 1115929,
"localTime" : ISODate("2014-07-07T15:27:14.416Z"),
"asserts" : {
"regular" : 0,
"warning" : 0,
"msg" : 0,
"user" : 2,
"rollovers" : 0
},
"backgroundFlushing" : {
"flushes" : 18843,
"total_ms" : 127648,
"average_ms" : 6.774292840842754,
"last_ms" : 2,
"last_finished" : ISODate("2014-07-07T15:26:43.282Z")
},
"connections" : {
"current" : 1,
"available" : 51199,
"totalCreated" : NumberLong(57)
},
"cursors" : {
"note" : "deprecated, use server status metrics",
"clientCursors_size" : 0,
"totalOpen" : 0,
"pinned" : 0,
"totalNoTimeout" : 2,
"timedOut" : 1
},
"extra_info" : {
"note" : "fields vary by platform",
"heap_usage_bytes" : 23483832,
"page_faults" : 13958
},
"globalLock" : {
"totalTime" : NumberLong("1130614310000"),
"lockTime" : NumberLong(184448),
"currentQueue" : {
"total" : 0,
"readers" : 0,
"writers" : 0
},
"activeClients" : {
"total" : 0,
"readers" : 0,
"writers" : 0
}
},
"indexCounters" : {
"accesses" : 451,
"hits" : 451,
"misses" : 0,
"resets" : 0,
"missRatio" : 0
},
"locks" : {
"." : {
"timeLockedMicros" : {
"R" : NumberLong(149),
"W" : NumberLong(184448)
},
"timeAcquiringMicros" : {
"R" : NumberLong(29),
"W" : NumberLong(32)
}
},
"admin" : {
"timeLockedMicros" : {
"r" : NumberLong(7348709),
"w" : NumberLong(0)
},
"timeAcquiringMicros" : {
"r" : NumberLong(55635),
"w" : NumberLong(0)
}
},
"local" : {
"timeLockedMicros" : {
"r" : NumberLong(59492773),
"w" : NumberLong(32)
},
"timeAcquiringMicros" : {
"r" : NumberLong(3164744),
"w" : NumberLong(3)
}
},
"config" : {
"timeLockedMicros" : {
"r" : NumberLong(182516),
"w" : NumberLong(0)
},
"timeAcquiringMicros" : {
"r" : NumberLong(46473),
"w" : NumberLong(0)
}
},
"mydb" : {
"timeLockedMicros" : {
"r" : NumberLong(43791920),
"w" : NumberLong(118)
},
"timeAcquiringMicros" : {
"r" : NumberLong(2159715),
"w" : NumberLong(9)
}
},
"test" : {
"timeLockedMicros" : {
"r" : NumberLong(28235652),
"w" : NumberLong(252)
},
"timeAcquiringMicros" : {
"r" : NumberLong(4052053),
"w" : NumberLong(19)
}
},
"mp3db" : {
"timeLockedMicros" : {
"r" : NumberLong(42491162),
"w" : NumberLong(1053565)
},
"timeAcquiringMicros" : {
"r" : NumberLong(6120501),
"w" : NumberLong(832)
}
}
},
"network" : {
"bytesIn" : 13516862,
"bytesOut" : 34014948,
"numRequests" : 733
},
"opcounters" : {
"insert" : 112,
"query" : 100247,
"update" : 0,
"delete" : 18,
"getmore" : 3,
"command" : 344
},
"opcountersRepl" : {
"insert" : 0,
"query" : 0,
"update" : 0,
"delete" : 0,
"getmore" : 0,
"command" : 0
},
"recordStats" : {
"accessesNotInMemory" : 10,
"pageFaultExceptionsThrown" : 1,
"admin" : {
"accessesNotInMemory" : 0,
"pageFaultExceptionsThrown" : 0
},
"config" : {
"accessesNotInMemory" : 0,
"pageFaultExceptionsThrown" : 0
},
"local" : {
"accessesNotInMemory" : 1,
"pageFaultExceptionsThrown" : 0
},
"mp3db" : {
"accessesNotInMemory" : 1,
"pageFaultExceptionsThrown" : 1
},
"mydb" : {
"accessesNotInMemory" : 2,
"pageFaultExceptionsThrown" : 0
},
"test" : {
"accessesNotInMemory" : 6,
"pageFaultExceptionsThrown" : 0
}
},
"writeBacksQueued" : false,
"mem" : {
"bits" : 32,
"resident" : 12,
"virtual" : 445,
"supported" : true,
"mapped" : 320
},
"metrics" : {
"cursor" : {
"timedOut" : NumberLong(1),
"open" : {
"noTimeout" : NumberLong(2),
"pinned" : NumberLong(0),
"total" : NumberLong(0)
}
},
"document" : {
"deleted" : NumberLong(72),
"inserted" : NumberLong(112),
"returned" : NumberLong(1294),
"updated" : NumberLong(0)
},
"getLastError" : {
"wtime" : {
"num" : 0,
"totalMillis" : 0
},
"wtimeouts" : NumberLong(0)
},
"operation" : {
"fastmod" : NumberLong(0),
"idhack" : NumberLong(0),
"scanAndOrder" : NumberLong(0)
},
"queryExecutor" : {
"scanned" : NumberLong(106),
"scannedObjects" : NumberLong(106)
},
"record" : {
"moves" : NumberLong(0)
},
"repl" : {
"apply" : {
"batches" : {
"num" : 0,
"totalMillis" : 0
},
"ops" : NumberLong(0)
},
"buffer" : {
"count" : NumberLong(0),
"maxSizeBytes" : 268435456,
"sizeBytes" : NumberLong(0)
},
"network" : {
"bytes" : NumberLong(0),
"getmores" : {
"num" : 0,
"totalMillis" : 0
},
"ops" : NumberLong(0),
"readersCreated" : NumberLong(0)
},
"preload" : {
"docs" : {
"num" : 0,
"totalMillis" : 0
},
"indexes" : {
"num" : 0,
"totalMillis" : 0
}
}
},
"storage" : {
"freelist" : {
"search" : {
"bucketExhausted" : NumberLong(0),
"requests" : NumberLong(97),
"scanned" : NumberLong(166)
}
}
},
"ttl" : {
"deletedDocuments" : NumberLong(0),
"passes" : NumberLong(18840)
}
},
"ok" : 1
}
>
dbStats - The dbStats command, or db.stats() from the shell, returns a document that addresses storage use and data volumes. The dbStats reflect the amount of storage used, the quantity of data contained in the database, and object, collection, and index counters.
> use mp3db
switched to db mp3db
> db.stats()
{
"db" : "mp3db",
"collections" : 4,
"objects" : 14,
"avgObjSize" : 42210.28571428572,
"dataSize" : 590944,
"storageSize" : 35037184,
"numExtents" : 7,
"indexes" : 4,
"indexSize" : 32704,
"fileSize" : 67108864,
"nsSizeMB" : 16,
"dataFileVersion" : {
"major" : 4,
"minor" : 5
},
"extentFreeList" : {
"num" : 0,
"totalSize" : 0
},
"ok" : 1
}
>
collStats - The collStats provides statistics that resemble dbStats on the collection level, including a count of the objects in the collection, the size of the collection, the amount of disk space used by the collection, and information about its indexes.
> db.mp3.chunks.stats()
{
"ns" : "mp3db.mp3.chunks",
"count" : 2,
"size" : 589792,
"avgObjSize" : 294896,
"storageSize" : 35012608,
"numExtents" : 4,
"nindexes" : 2,
"lastExtentSize" : 15290368,
"paddingFactor" : 1,
"systemFlags" : 1,
"userFlags" : 1,
"totalIndexSize" : 16352,
"indexSizes" : {
"_id_" : 8176,
"files_id_1_n_1" : 8176
},
"ok" : 1
}
>
replSetGetStatus - The replSetGetStatus command (rs.status() from the shell) returns an overview of your replica set’s status. The replSetGetStatus document details the state and configuration of the replica set and statistics about its members.
> rs.status()
{ "ok" : 0, "errmsg" : "not running with --replSet" }
log available in /var/log/mongodb/mongod.log
configuration
There are too many configurations to covered here but below are the essential configurations which you might need to change.
As mentioned previously, if the app that connected to the database are on two different servers, then in the server that run mongo instance, you should comment out bind_ip
# Listen to local interface only. Comment out to listen on all interfaces.
#bind_ip = 127.0.0.1
Default port running is 27017 but you should make sure that this port and ip allow to be access remotely.
use kernel 2.6.36 or later.
In general, if you use the Ext4 file system, use at least version 2.6.23 of the Linux Kernel.
In general, if you use the XFS file system, use at least version 2.6.25 of the Linux Kernel.
Set the file descriptor limit, -n, and the user process limit (ulimit), -u, above 20,000, according to the suggestions in the ulimit document. A low ulimit will affect MongoDB when under heavy use and can produce errors and lead to failed connections to MongoDB processes and loss of service.
That's it. Please go to the donate page and contribute back if you learned something.
No comments:
Post a Comment