MongoDB 数据文件打包姿势不对引起灾难!

节前,想着要给这个站点做数据备份了。

但是一时脑残想偷懒,所以进行了一次很直接的备份。


就是直接对 MongoDB 正在运行的数据文件进行了打包操作。

打包完后,发现 MongoDB 服务起不来了,运行命令:

mongod &

核心报错错误信息:

10334:BSONObj size: 0 (0x0) is invalid. Size must be between 0 and 16793600(16MB) First element: EOO

开始以为是服务器版本有问题,于是把打包好的数据文件拿到本地机器调试。


关闭 MongoDB 服务,覆盖本地的开发库数据文件,启动 MongoDB 服务。

很不幸,同样的错误提示:

2015-09-26T16:33:59.743 0800 I CONTROL  Trying to start Windows service "MongoDB"
2015-09-26T16:33:59.744 0800 I STORAGE  Service running
2015-09-26T16:33:59.754 0800 I JOURNAL  [initandlisten] journal dir=D:/Database/MongoDB/Data/journal
2015-09-26T16:33:59.754 0800 I JOURNAL  [initandlisten] recover : no journal files present, no recovery needed
2015-09-26T16:33:59.822 0800 I JOURNAL  [durability] Durability thread started
2015-09-26T16:33:59.823 0800 I JOURNAL  [journal writer] Journal writer thread started
2015-09-26T16:33:59.946 0800 I CONTROL  [initandlisten] MongoDB starting : pid=6884 port=11111 dbpath=D:/Database/MongoDB/Data 64-bit host=dev-ttt
2015-09-26T16:33:59.947 0800 I CONTROL  [initandlisten] targetMinOS: Windows 7/Windows Server 2008 R2
2015-09-26T16:33:59.947 0800 I CONTROL  [initandlisten] db version v3.0.1
2015-09-26T16:33:59.947 0800 I CONTROL  [initandlisten] git version: 534b5a3f9d10f00cd27737fbcd951032248b5952
2015-09-26T16:33:59.947 0800 I CONTROL  [initandlisten] OpenSSL version: OpenSSL 1.0.1j-fips 15 Oct 2014
2015-09-26T16:33:59.947 0800 I CONTROL  [initandlisten] build info: windows sys.getwindowsversion(major=6, minor=1, build=7601, platform=2, service_pack="Service Pack 1") BOOST_LIB_VERSION=1_49
2015-09-26T16:33:59.947 0800 I CONTROL  [initandlisten] allocator: system
2015-09-26T16:33:59.947 0800 I CONTROL  [initandlisten] options: { service: true, storage: { dbPath: "D:/Database/MongoDB/Data" }, systemLog: { destination: "file", path: "D:/Database/MongoDB/Log/db.log" } }
2015-09-26T16:33:59.952 0800 I -        [initandlisten] Assertion: 10334:BSONObj size: 0 (0x0) is invalid. Size must be between 0 and 16793600(16MB) First element: EOO
2015-09-26T16:34:00.535 0800 I CONTROL  [initandlisten] mongod.exe    .../src/mongo/util/stacktrace_win.cpp(175)                                           mongo::printStackTrace 0x43
2015-09-26T16:34:00.535 0800 I CONTROL  [initandlisten] mongod.exe    .../src/mongo/util/log.cpp(135)                                                      mongo::logContext 0x97
2015-09-26T16:34:00.535 0800 I CONTROL  [initandlisten] mongod.exe    .../src/mongo/util/assert_util.cpp(214)                                              mongo::msgasserted 0xd7
2015-09-26T16:34:00.535 0800 I CONTROL  [initandlisten] mongod.exe    .../src/mongo/util/assert_util.cpp(206)                                              mongo::msgasserted 0x13
2015-09-26T16:34:00.535 0800 I CONTROL  [initandlisten] mongod.exe    .../src/mongo/bson/bsonobj.cpp(73)                                                   mongo::BSONObj::_assertInvalid 0x40d
2015-09-26T16:34:00.535 0800 I CONTROL  [initandlisten] mongod.exe    .../src/mongo/db/storage/record_data.h(66)                                           mongo::RecordData::toBson 0xa4
2015-09-26T16:34:00.535 0800 I CONTROL  [initandlisten] mongod.exe    .../src/mongo/db/storage/mmap_v1/catalog/namespace_details_collection_entry.cpp(97)  mongo::NamespaceDetailsCollectionCatalogEntry::getAllIndexes 0x100
2015-09-26T16:34:00.535 0800 I CONTROL  [initandlisten] mongod.exe    .../src/mongo/db/storage/mmap_v1/mmap_v1_database_catalog_entry.cpp(632)             mongo::MMAPV1DatabaseCatalogEntry::_init 0xb22
2015-09-26T16:34:00.535 0800 I CONTROL  [initandlisten] mongod.exe    .../src/mongo/db/storage/mmap_v1/mmap_v1_database_catalog_entry.cpp(185)             mongo::MMAPV1DatabaseCatalogEntry::MMAPV1DatabaseCatalogEntry 0x262
2015-09-26T16:34:00.535 0800 I CONTROL  [initandlisten] mongod.exe    .../src/mongo/db/storage/mmap_v1/mmap_v1_engine.cpp(272)                             mongo::MMAPV1Engine::getDatabaseCatalogEntry 0x1a4
2015-09-26T16:34:00.535 0800 I CONTROL  [initandlisten] mongod.exe    .../src/mongo/db/catalog/database_holder.cpp(128)                                    mongo::DatabaseHolder::openDb 0x242
2015-09-26T16:34:00.535 0800 I CONTROL  [initandlisten] mongod.exe    .../src/mongo/db/db.cpp(372)                                                         mongo::repairDatabasesAndCheckVersion 0x42f
2015-09-26T16:34:00.536 0800 I CONTROL  [initandlisten] mongod.exe    .../src/mongo/db/db.cpp(540)                                                         mongo::_initAndListen 0xb71
2015-09-26T16:34:00.536 0800 I CONTROL  [initandlisten] mongod.exe    .../src/mongo/db/db.cpp(641)                                                         mongo::initAndListen 0x27
2015-09-26T16:34:00.536 0800 I CONTROL  [initandlisten] mongod.exe    .../src/mongo/db/db.cpp(666)                                                         mongo::initService 0x85
2015-09-26T16:34:00.536 0800 I CONTROL  [initandlisten] mongod.exe    .../src/mongo/util/ntservice.cpp(524)                                                mongo::ntservice::initService 0x50
2015-09-26T16:34:00.536 0800 I CONTROL  [initandlisten] sechost.dll                                                                                        SubscribeServiceChangeNotifications 0x2ca
2015-09-26T16:34:00.536 0800 I CONTROL  [initandlisten] KERNEL32.DLL                                                                                       BaseThreadInitThunk 0x22
2015-09-26T16:34:00.536 0800 I CONTROL  [initandlisten]
2015-09-26T16:34:00.536 0800 W STORAGE  [initandlisten] database D:/Database/MongoDB/Data Blog could not be opened due to DBException 10334: BSONObj size: 0 (0x0) is invalid. Size must be between 0 and 16793600(16MB) First element: EOO
2015-09-26T16:34:00.538 0800 I STORAGE  [initandlisten] exception in initAndListen: 10334 BSONObj size: 0 (0x0) is invalid. Size must be between 0 and 16793600(16MB) First element: EOO, terminating
2015-09-26T16:34:00.538 0800 I CONTROL  [initandlisten] now exiting
2015-09-26T16:34:00.538 0800 I NETWORK  [initandlisten] shutdown: going to close listening sockets...
2015-09-26T16:34:00.538 0800 I NETWORK  [initandlisten] shutdown: going to flush diaglog...
2015-09-26T16:34:00.538 0800 I NETWORK  [initandlisten] shutdown: going to close sockets...
2015-09-26T16:34:00.539 0800 I STORAGE  [initandlisten] shutdown: waiting for fs preallocator...
2015-09-26T16:34:00.539 0800 I STORAGE  [initandlisten] shutdown: final commit...
2015-09-26T16:34:00.539 0800 I JOURNAL  [initandlisten] journalCleanup...
2015-09-26T16:34:00.546 0800 I JOURNAL  [initandlisten] removeJournalFiles
2015-09-26T16:34:00.547 0800 I JOURNAL  [initandlisten] Terminating durability thread ...
2015-09-26T16:34:00.642 0800 I JOURNAL  [journal writer] Journal writer thread stopped
2015-09-26T16:34:00.642 0800 I JOURNAL  [durability] Durability thread stopped
2015-09-26T16:34:00.642 0800 I STORAGE  [initandlisten] shutdown: closing all files...
2015-09-26T16:34:00.642 0800 I STORAGE  [initandlisten] closeAllFiles() finished
2015-09-26T16:34:00.642 0800 I STORAGE  [initandlisten] shutdown: removing fs lock...
2015-09-26T16:34:00.642 0800 I CONTROL  [initandlisten] dbexit:  rc: 49

想来问题还是出在打包操作上。

估计是因为直接对正在运行的文件进行打包,导致数据文件损坏。


使用 --repair 操作也没办法修复数据。

最后,只能拿了最近备份的一个完好数据提交到生产环境进行文件覆盖,然后使用 --repair 操作修复后,重新启动 MongoDB 服务,服务正常启动运行。


真是严重的灾难事故!!!

以后还是不能这样违规操作了!!!