A MongoDB 3.6.3 database has two shards and a mongodump file with partial data of one collection needs to be restored. The first shard is named "fast" while second "slow". The idea is to restore the dump to the "slow" shard. According to sharding rules the data should go to the "slow" shard, but it actually goes to the wrong one when restore is tried. Before restoring the data, I want to manually move a range of chunks from fast to slow shard, but are unable to properly issue the command. All examples found are showing of moving only one exact chunk. _id is used as a sharding key. Try 1: use admin db.runCommand({ moveChunk: "db.use", bounds : [ {_id : ObjectId("58b60e73e5d4e7019aa2be17")}, {_id : ObjectId("58bca60f5067031c77b03807")} ], to: "rs1" }) This is the response: { "ok" : 0, "errmsg" : "no chunk found with the shard key bounds [{ _id: ObjectId('58b60e73e5d4e7019aa2be17') }, { _id: ObjectId('58bca60f5067031c77b03807') })", "$clusterTime" : { "clusterTime" : Timestamp(1523459407, 14673), "signature" : { "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="), "keyId" : NumberLong(0) } }, "operationTime" : Timestamp(1523459407, 14673) } Try 2: sh.moveChunk("db.use", {_id:{$gt: ObjectId("58b60e73e5d4e7019aa2be17"), $lt: ObjectId("58bca60f5067031c77b03807") }},"rs1") Response: { "ok" : 0, "errmsg" : "no shard key found in chunk query { _id: { $gt: ObjectId('58b60e73e5d4e7019aa2be17'), $lt: ObjectId('58bca60f5067031c77b03807') } }", "$clusterTime" : { "clusterTime" : Timestamp(1523460271, 11742), "signature" : { "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="), "keyId" : NumberLong(0) } }, "operationTime" : Timestamp(1523460271, 11742) } Any idea how to move chunks that belong to certain shard key range?

A MongoDB v3.6.4 database with a large sharded collection and WiredTiger engine hangs when issuing a findOne() or find() command without any parameters. The collection has only the default index on the _id key. If findOne() or find() is invoked on any of the shards then it works immediately. If it is invoked on a mongos then it hangs. Why this query hangs? By looking with db.currentOp() MongoDB uses COLLSCAN. The funny thing is that this worked and then suddenly stopped yesterday. As a note, there is a sharding migration process which moves chunks, but that doesn't seem to influence as even when chunk balancing is stopped or when it is finished, still findOne() hangs (or doesn't hang but scans all the collection). Here is the output of db.currentOp() taken from one primary: { "host" : "db1:27017", "desc" : "conn277", "connectionId" : 277, "client" : "10.240.137.10:57294", "appName" : "MongoDB Shell", "clientMetadata" : { "application" : { "name" : "MongoDB Shell" }, "driver" : { "name" : "MongoDB Internal Client", "version" : "3.6.4" }, "os" : { "type" : "Linux", "name" : "PRETTY_NAME=\"Debian GNU/Linux 9 (stretch)\"", "architecture" : "x86_64", "version" : "Kernel 4.9.0-6-amd64" }, "mongos" : { "host" : "m1:27017", "client" : "10.240.0.0:38190", "version" : "3.6.4" } }, "active" : true, "currentOpTime" : "2018-04-28T08:13:26.371+0200", "opid" : 2148656, "secs_running" : NumberLong(4), "microsecs_running" : NumberLong(4344989), "op" : "query", "ns" : "somedb.somecol", "command" : { "find" : "somecol", "limit" : NumberLong(1), "shardVersion" : [ Timestamp(188696, 1), ObjectId("5ac6b2abbd8bbc9f42f34a39") ], "$clusterTime" : { "clusterTime" : Timestamp(1524896001, 1), "signature" : { "hash" : BinData(0,"RuV/v6Qm7H9AvPMVRNH0jkdIwRM="), "keyId" : NumberLong("6540622458888650772") } }, "$client" : { "application" : { "name" : "MongoDB Shell" }, "driver" : { "name" : "MongoDB Internal Client", "version" : "3.6.4" }, "os" : { "type" : "Linux", "name" : "PRETTY_NAME=\"Debian GNU/Linux 9 (stretch)\"", "architecture" : "x86_64", "version" : "Kernel 4.9.0-6-amd64" }, "mongos" : { "host" : "m1:27017", "client" : "10.240.0.0:38190", "version" : "3.6.4" } }, "$configServerState" : { "opTime" : { "ts" : Timestamp(1524896001, 1), "t" : NumberLong(3) } }, "$db" : "feedback" }, "planSummary" : "COLLSCAN", "numYields" : 803, "locks" : { "Global" : "r", "Database" : "r", "Collection" : "r" }, "waitingForLock" : false, "lockStats" : { "Global" : { "acquireCount" : { "r" : NumberLong(1608) } }, "Database" : { "acquireCount" : { "r" : NumberLong(804) } }, "Collection" : { "acquireCount" : { "r" : NumberLong(804) } } } },

I have setup a minimal replicated configuration with mongodb 3.6.3,it seems to be everything fine but when I connect to the mongos (which as far as I understand acts like a kind of load balancer in front of sharded clusters) and try to perform a query it raises this error: *Cannot accept sharding commands if not started with --shardsvr* (code: 193, codename: NoShardingEnabled) This is how I've setup the servers: 3 servers: mongoserver1, mongoserver2 and mongoserver3 Launch the config server: mongod --configsvr --replSet data --bind_ip localhost,${HOSTNAME}\ --port 25000 --logpath /var/lib/mongodb/shardedcluster/log.cfg\ --logappend --dbpath /var/lib/mongodb/shardedcluster/cfg --fork Log in the config server and initiate the replica: rs.initiate( { _id: "data", configsvr: true, members: [ { _id : 0, host : "mongoserver1:25000" }, { _id : 1, host : "mongoserver2:25000" }, { _id : 2, host : "mongoserver3:25000" } ] } ) Launch the sharded replica: mongod --shardsvr --replSet data --bind_ip localhost,${HOSTNAME}\ --port 26000 --logpath /var/lib/mongodb/shardedcluster/data.log\ --logappend --dbpath /var/lib/mongodb/shardedcluster/data --fork Log in the sharde replica and initiate the replica: rs.initiate( { _id : "data", members: [ { _id : 0, host : "mongoserver1:26000" }, { _id : 1, host : "mongoserver2:26000" }, { _id : 2, host : "mongoserver3:26000" } ] } ) And finally launched the mongos: mongos --configdb data/mongoserver1:25000,mongoserver2:25000,mongoserver3:25000\ --bind_ip localhost,${HOSTNAME}\ --logpath /var/lib/mongodb/shardedcluster/mongos.log --fork And inside the mongos: sh.addShard("data/mongoserver1:26000,mongoserver2:26000,mongoserver3:26000") sh.enableSharding("cmdb") After adding some data on the cmdb database through the mongod running or port 26000 I've checked after stopping primary node I can continue working with date on the new primary node but I'm not able to do it connecting to mongos even it seems the cmdb database is replicated as shown by sh.status(): databases: { "_id" : "cmdb", "primary" : "data", "partitioned" : true }

I have migrated the collection.bson file from **MongoDB 2.4** to **MongoDB 3.6.4** as collection.json. But their is 1 more file collection.metadata.json in **MongoDB 2.4**. How can I import this to **MongoDB 3.6.4**? content of collection.metadata.json is as below; { "options": { "create": "collection" }, "indexes": [ { "v": 1, "name": "_id_", "key": { "_id": 1 }, "ns": "emgda.collection" }, { "v": 1, "name": "customerId_1_elementId_1_siteId_1_createdTimestamp_1", "key": { "customerId": 1, "elementId": 1, "siteId": 1, "createdTimestamp": 1 }, "ns": "emgda.collection", "background": true, "sparse": true } ] } > mongorestore version is as below, emgda@ubuntu:~/trial$ mongorestore --version mongorestore version: r3.6.4 git version: d0181a711f7e7f39e60b5aeb1dc7097bf6ae5856 Go version: go1.7 os: linux arch: amd64 compiler: gc OpenSSL version: OpenSSL 1.0.2g 1 Mar 2016 > NOTE: When I use mongoretore -d emgda auditing.bson, I get below error for the collection. checking for collection data in auditing.bson reading metadata for emgda.auditing from auditing.metadata.json Failed: emgda.auditing: error creating collection emgda.auditing: error running create command: BSON field 'OperationSessionInfo.create' is a duplicate field

I have a MongoDB schema that looks like this: const batchSchema = mongoose.Schema({ _id: mongoose.Schema.Types.ObjectId, time: {type: [Number]}, tmp: {type: [Number]}, hum: {type: [Number]}, co2: {type: [Number]}, coolRelay: {type: [Boolean]}, humRelay: {type: [Boolean]}, fanRelay: {type: [Boolean]}, lightRelay: {type: [Boolean]}, }); My goal is to have a csv file I can import to excel for use in creating charts, graphs, and other visuals based on the data. Using Studio 3T(https://robomongo.org/) , this doesn't seem to be possible unless I'm just not understanding the settings. Doesn't really matter if the data is organized horizontally or vertically. As long as each is on it own column/row, excluding the _id field. 3T seems to be only capable of placing them all along the same row. I just tried using mongo's native cli csv export tool but found that didn't seem to come with my release. I'm running version 3.6 on openSuse, arm64, on a raspberry pi. mongoexport as a command was not found, and when I ran the suggested cnf mongoexport to find a package containing it, nothing was returned either. I know that I can export one file, per array using 3T, then cut/paste them together, however I hadn't planned on paying the annual license for 3T and was only using the free trial to get a more usable gui while in development. Was hoping to be able to write predefined bash script that could be reused repeatedly to export each "batch" document as it was completed. Is there something I am missing about mongoexport or is this a limitation of the aarch64 release? I really don't want to have to go back, learn, and re-write everything for postgreSQL at this point. I mean, I will if I have to, but wanted to check in with you all first.

I've been trying to understand the MongoDB Aggregate process so I can better optimize my queries and I'm confused by usage and $match and $sort together. Sample DB has only one collection people [{ "name": "Joe Smith", "age": 40, "admin": false }, { "name": "Jen Ford", "age": 45, "admin": true }, { "name": "Steve Nash", "age": 45, "admin": true }, { "name": "Ben Simmons", "age": 45, "admin": true }] I've multiplied this data x1000 just as a POC. The DB above has one index name_1 The Following query db.people.find({"name": "Jen Ford"}).sort({"_id": -1}).explain() Has the following output { queryPlanner: { plannerVersion: 1, namespace: 'db.people', indexFilterSet: false, parsedQuery: { name: { '$eq': 'Jen Ford' } }, queryHash: '3AE4BDA3', planCacheKey: '2A9CC473', winningPlan: { stage: 'SORT', sortPattern: { _id: -1 }, inputStage: { stage: 'SORT_KEY_GENERATOR', inputStage: { stage: 'FETCH', inputStage: { stage: 'IXSCAN', keyPattern: { name: 1 }, indexName: 'name_1', isMultiKey: false, multiKeyPaths: { name: [] }, isUnique: false, isSparse: false, isPartial: false, indexVersion: 2, direction: 'forward', indexBounds: { name: [ '["Jen Ford", "Jen Ford"]' ] } } } } }, rejectedPlans: [ { stage: 'FETCH', filter: { name: { '$eq': 'Jen Ford' } }, inputStage: { stage: 'IXSCAN', keyPattern: { _id: 1 }, indexName: '_id_', isMultiKey: false, multiKeyPaths: { _id: [] }, isUnique: true, isSparse: false, isPartial: false, indexVersion: 2, direction: 'backward', indexBounds: { _id: [ '[MaxKey, MinKey]' ] } } } ] }, serverInfo: { host: '373ea645996b', port: 27017, version: '4.2.0', gitVersion: 'a4b751dcf51dd249c5865812b390cfd1c0129c30' }, ok: 1 } This makes total sense. **However** The following query results in the same set but uses the aggregate pipeline db.people.aggregate([ { $match: { $and: [{ name: "Jen Ford" }]}}, { $sort: {"_id": -1}}], {"explain": true}) Has the following output. { queryPlanner: { plannerVersion: 1, namespace: 'db.people', indexFilterSet: false, parsedQuery: { name: { '$eq': 'Jen Ford' } }, queryHash: '3AE4BDA3', planCacheKey: '2A9CC473', optimizedPipeline: true, winningPlan: { stage: 'FETCH', filter: { name: { '$eq': 'Jen Ford' } }, inputStage: { stage: 'IXSCAN', keyPattern: { _id: 1 }, indexName: '_id_', isMultiKey: false, multiKeyPaths: { _id: [] }, isUnique: true, isSparse: false, isPartial: false, indexVersion: 2, direction: 'backward', indexBounds: { _id: [ '[MaxKey, MinKey]' ] } } }, rejectedPlans: [] }, serverInfo: { host: '373ea645996b', port: 27017, version: '4.2.0', gitVersion: 'a4b751dcf51dd249c5865812b390cfd1c0129c30' }, ok: 1 } **Notice how the Aggregate Query is unable to recognize it should utilize the name index against the $match. This has massive implications as the size of the collection grows** I've seen this behavior now in Mongo 3.4, 3.6, and 4.2. https://docs.mongodb.com/v4.2/core/aggregation-pipeline-optimization/ provides this blurb > $sort + $match Sequence Optimization: When you have a sequence with $sort followed by a $match, the $match moves before the $sort to minimize the number of objects to sort. From all this, I think I'm fundamentally misunderstanding something with the Mongo aggregate command. I already understand that if I create a composite index name,_id then it will work as it includes the fields used in my $match and my $sort clause. But **why** must an index include a field from the $sort clause to be utilized to restrict my $match set? It seems obvious that we would prefer to $sort on the smallest set possible?

My query with $and, $or operators is performing poorly. Even though I've built a compound index on all three fields that are specified in my query criteria, the query does a complete collection scan. I have tried both simple find and aggregation and the Winning plan for both is COLLSCAN. ### Simple find Query:

db.test.find({ $or: [ {id1: "900004"}, {relatedid: "900004"}], $and: [ { isValid: "Y" } ] } )

Execution stats:

{
    "queryPlanner" : {
        "plannerVersion" : 1,
        "namespace" : "XXX",
        "indexFilterSet" : false,
        "parsedQuery" : {
            "$and" : [ 
                {
                    "$or" : [ 
                        {
                            "Id1" : {
                                "$eq" : "900004"
                            }
                        }, 
                        {
                            "relatedID" : {
                                "$eq" : "900004"
                            }
                        }
                    ]
                }, 
                {
                    "isValid" : {
                        "$eq" : "Y"
                    }
                }
            ]
        },
        "winningPlan" : {
            "stage" : "COLLSCAN",
            "filter" : {
                "$and" : [ 
                    {
                        "$or" : [ 
                            {
                                "Id1" : {
                                    "$eq" : "900004"
                                }
                            }, 
                            {
                                "relatedID" : {
                                    "$eq" : "900004"
                                }
                            }
                        ]
                    }, 
                    {
                        "isValid" : {
                            "$eq" : "Y"
                        }
                    }
                ]
            },
            "direction" : "forward"
        },
        "rejectedPlans" : []
    },
    "executionStats" : {
        "executionSuccess" : true,
        "nReturned" : 1,
        "executionTimeMillis" : 61,
        "totalKeysExamined" : 0,
        "totalDocsExamined" : 100006,
        "executionStages" : {
            "stage" : "COLLSCAN",
            "filter" : {
                "$and" : [ 
                    {
                        "$or" : [ 
                            {
                                "Id1" : {
                                    "$eq" : "900004"
                                }
                            }, 
                            {
                                "relatedID" : {
                                    "$eq" : "900004"
                                }
                            }
                        ]
                    }, 
                    {
                        "isValid" : {
                            "$eq" : "Y"
                        }
                    }
                ]
            },
            "nReturned" : 1,
            "executionTimeMillisEstimate" : 60,
            "works" : 100008,
            "advanced" : 1,
            "needTime" : 100006,
            "needYield" : 0,
            "saveState" : 781,
            "restoreState" : 781,
            "isEOF" : 1,
            "invalidates" : 0,
            "direction" : "forward",
            "docsExamined" : 100006
        }
    },
    "serverInfo" : {
        "host" : "XXXX",
        "port" : ,
        "version" : "4.0.9",
        "gitVersion" : "XXX"
    },
    "ok" : 1.0
}

### Aggregation Query:

db.test.aggregate( [{ "$match":{ "$and": [ {"isValid": "Y"}, { "$or": [ {"Id1": "900004"}, {"relatedID": "900004"} ]}] }} ] )

Execution Stats:

{
    "stages" : [ 
        {
            "$cursor" : {
                "query" : {
                    "$and" : [ 
                        {
                            "isValid" : "Y"
                        }, 
                        {
                            "$or" : [ 
                                {
                                    "Id1" : "900004"
                                }, 
                                {
                                    "relatedID" : "900004"
                                }
                            ]
                        }
                    ]
                },
                "queryPlanner" : {
                    "plannerVersion" : 1,
                    "namespace" : "XXXXX",
                    "indexFilterSet" : false,
                    "parsedQuery" : {
                        "$and" : [ 
                            {
                                "$or" : [ 
                                    {
                                        "Id1" : {
                                            "$eq" : "900004"
                                        }
                                    }, 
                                    {
                                        "relatedID" : {
                                            "$eq" : "900004"
                                        }
                                    }
                                ]
                            }, 
                            {
                                "isValid" : {
                                    "$eq" : "Y"
                                }
                            }
                        ]
                    },
                    "winningPlan" : {
                        "stage" : "COLLSCAN",
                        "filter" : {
                            "$and" : [ 
                                {
                                    "$or" : [ 
                                        {
                                            "Id" : {
                                                "$eq" : "900004"
                                            }
                                        }, 
                                        {
                                            "relatedID" : {
                                                "$eq" : "900004"
                                            }
                                        }
                                    ]
                                }, 
                                {
                                    "isValid" : {
                                        "$eq" : "Y"
                                    }
                                }
                            ]
                        },
                        "direction" : "forward"
                    },
                    "rejectedPlans" : []
                }
            }
        }
    ],
    "ok" : 1.0
}

How can I get a fast a query for my results?

I am trying to export collection data from (SSL MongoDB 4.x), for this I am running below command on linux terminal. Earlier I used mongoexport command in Linux terminal without SSL which works properly for (MongoDB without SSL). sudo mongoexport --verbose=1 --host X.X.X.X:27018 --ssl --sslCAFile "certificate.pem" -u username -p password --authenticationDatabase XYZ_Database --collection XYZ_Collection --out Output.csv ##Error 2020-01-28T15:15:03.266+0000 could not connect to server: server selection error: server selection timeout current topology: Type: Single Servers: Addr: X.X.X.X:27018, Type: Unknown, State: Connected, Average RTT: 0, Last error: connection(X.X.X.X:27018[-121]) connection is closed

This is what I want to do: 1. Create a Config Server 2. Create 2 Shard 3. Create a Mongos Router I don't want to replicate my data using replica set, I want to simulate that each shard has only a part of data like it says is shard theory. I'm following this guide , but I understood that the guide create many replicas for config server and shared server and I don't want that. So this is what I do: Config file of Config Server: sharding: clusterRole: configsvr replication: replSetName: localhost net: bindIp: localhost storage: dbPath: /home/christian/mongo/confsvr/metadata Execute Config Server: mongod --config /mongo/confsvr/mongod.conf --port 27019 Now doc uses rs.initiate() for adding a replica of config server (if I understood right), so I dropped this part. After that, I created two config file for Shard Server. sharding: clusterRole: shardsvr replication: replSetName: localhost net: bindIp: localhost storage: dbPath: /home/christian/mongo/shard[1|2] I started two shard: mongod --config mongo/shard[1|2]/shard.conf --port 2703[1|2] Created conf file for Mongos Router: sharding: configDB:localhost/localhost:27019 net: bindIp: localhost So i tried to start mongos: mongos --config /home/christian/mongo/router/mongos.conf But terminal print in loop this: >Unable to reach primary for set localhost > >Cannot Reach any nodes for set localhost Last steps were connecting to mongos, and add the Shards to DB How could I obtain what I want?

On a MongoDB 3.6 in a replica set and WiredTiger engine we have a large collection with 7.7TB of data. We started to see severe performance issues (extreme CPU usage and slowdown of data replication) which we somehow solved by temporary using a new empty database. The data is basically usage logs and DB pattern is that mostly there are inserts (of new usage data) and occasionally data reads for certain users or reports. To scale the system and solve performance issues in the long run, we now created two shards where the idea is to have a fast shard (with data of recent 2 months) and a slow shard (with all historical data). Fast shard run on n1-standard-8 (8 vCPUs, 30 GB memory) machines with SSD disks, while slow shard is on n1-highmem-2 (2 vCPUs, 13 GB memory) machines with rotational disks. The database is not under any load (there are no new usage log inserts) and basically does nothing. The sharding process has started, but the speed of moving chunks is slow and will take around 40 days to move all the chunks. There are 200000 chunks left to be moved from *fast* to *slow* shard with average chunk size 32MB and it takes about 18 seconds to move one chunk. What can be done to speed up this chunk migration process? **UPDATE** I have learned that if source shard is on SSD disks, then the chunk moving has almost double speed.

Have setup a 4 node test shard cluster: * 1- mongos * 1- config server * 2- shard nodes hosting 1 shard each for a collection. Cluster is working perfectly, but when trying to run the following command: mongos> sh.addTagRange("HTMLDumps.HTMLRepository", { ShardMapId: "7" }, { ShardMapId: "11" }, "NRT") "errmsg" : "min: { ShardMapId: \"7\" } should be less than max: { ShardMapId: \"11\" }", "code" : 9, "codeName" : "FailedToParse", Below is the shard status output: databases: { "_id" : "HTMLDumps", "primary" : "shard0001", "partitioned" : true } HTMLDumps.HTMLRepository shard key: { "ShardMapId" : 1 } unique: false balancing: true chunks: shard0000 2 shard0001 2 { "ShardMapId" : { "$minKey" : 1 } } -->> { "ShardMapId" : "" } on : shard0001 Timestamp(3, 1) { "ShardMapId" : "" } -->> { "ShardMapId" : "1" } on : shard0001 Timestamp(1, 2) jumbo { "ShardMapId" : "1" } -->> { "ShardMapId" : "7" } on : shard0000 Timestamp(2, 0) { "ShardMapId" : "7" } -->> { "ShardMapId" : { "$maxKey" : 1 } } on : shard0000 Timestamp(3, 0) tag: NYC { "ShardMapId" : "1" } -->> { "ShardMapId" : "7" } ----------------------------- What am I missing? ------------------------------- Complete sh.status output:

We're running a huge MongoDB with a large index (made of of four fields) which was created ad-hoc and should now be superseded by slimmer one (of the first two fields). But the problem is we can't just delete the old one as the warmup of the new one may take hours (or days!). We have two secondary instances, is it possible to only delete the large index on one of those and have that instance take over when the new index is warmed? Or is there some way to guide Mongo to the new indices? **Edit:** on a small test database Mongo rejects the new index after I delete the old one and instead uses half of third index, even though the new one is a perfect fit for the query. How to guide/update the query plan?

I have two MongoDB clusters: DEV and PRO. For backup purposes, I use mongodump to export each and every database from the PRO cluster, including users, indexes, and roles:

mongodump --host "" --dumpDbUsersAndRoles ...

When I restore the database example1 from the PRO cluster into the DEV cluster, everything works as expected: all collections, documents, users, and indexes are restored correctly.

mongorestore -d example1 --restoreDbUsersAndRoles --host "" backup_pro_example1.tar.gz

However, if I try to restore the backup under a **different database name**, e.g: example2, while collections and documents are restored correctly, **users and indexes are missing**.

mongorestore -d example2 --restoreDbUsersAndRoles --host "" backup_pro_example1.tar.gz

How can I work around this issue to ensure users and indexes are also restored when restoring to a different database name?

I'm currently experimenting with upgrading mongo, comparing performance of different mongodb versions and stumbled into a curious anomality when upgrading from 3.4 to 3.6. I have a relatively large collection with an index that looks like this:

{ a: 1, 'array.b': 1, 'array.0.date': -1 }

And the following query:

db.myCollection.find({
    "array.b" : ObjectId(...),
    "a" : ObjectId(...)
}, { ... bunch of fields }).sort({ "array.0.date" : -1 }).skip(0).limit(26);

In 3.4 I see that Mongo reliably uses the proper index and explain shows that it uses IXSCAN as a first step with exactly 26 works, as I would expect. In 3.6, for some reason (I didn't find explanation) explain returns 2 rows. The 1st one looks exactly like the one used by 3.4, and the 2nd row shows that 1st IXSCAN returns all documents matching first two fields in the index resulting in a regular SORT being performed. It wouldn't bother me if without explain mongo would end up using 1st row which would be an optimal way, but in reality ends up using an in-memory sort. Is it possible to make 3.6 use specific execution plan (returned as the 1st row of explain)? Thanks! Update 1: found a relevant note in blue here (https://www.mongodb.com/docs/v3.6/tutorial/sort-results-with-indexes/) , going to test if moving array fields to the top level would help, array.b is also an array, so who knows...

I really like using mongodb to store my data and recently I tried out GridFS and it really fits my use case. My problem with it is the space requirement, which seems quite odd. I have ~107GB of images in Amazon S3, which is around 1 million files (all images, mostly small ones). I made a simple Java project to download the images from S3 and insert them into two separate MongoDB GridFS collections (single server, 3.6.5, 64 bit, Windows Server 2016). The problem is, when the upload/download completes, the GridFS collections take more than 300GB storage on the server. Is this acceptable for this kind of collection or should I worry about the tripled size? Note: I simply inserted the images using the Java Mongo Driver (Spring Boot) without any significant change, the problem is with the image chunks. I do not delete or update any images (I defined a unique index for the MD5 field though, to ignore image duplication), thus compact and repair does not change the collection sizes. As much as I can see, the collections are not overly preallocated (I don't think my problem is similar to this: https://dba.stackexchange.com/questions/68370/huge-size-on-mongodbs-gridfs-should-i-compact ) Also, currently it is a single mongodb server, without a replica set. Thank you very much for your help!

I'm using mongodb 3.6.18 I'm running a cluster of 2 shards and 1 config replicaset. I get this error when trying to set the compatibility version to 3.6: global:PRIMARY> db.adminCommand( { setFeatureCompatibilityVersion: "3.6" } ) { "ok" : 0, "errmsg" : "expected entry { _id: \"son.service_state\", lastmodEpoch: ObjectId('5f743676f790314e2a02cfd3'), lastmod: new Date(4294967296), dropped: false, key: { data_center: 1 }, unique: false } in config.collections for son.service_state to have a UUID, but it did not", "code" : 1, "codeName" : "InternalError", "$gleStats" : { "lastOpTime" : { "ts" : Timestamp(1601465931, 1), "t" : NumberLong(1) }, "electionId" : ObjectId("7fffffff0000000000000001") }, "$configServerState" : { "opTime" : { "ts" : Timestamp(1601465922, 1), "t" : NumberLong(1) } } } Did anyone encounter this before? how can i fix this?

A two shards MongoDB database regularly crashes with out-of-memory error or is being killed by the oom-killer. The system runs on **GCE Debian 9.4** with **MongoDB v3.6.5**, **WiredTiger** storage engine and without swap (as is the practice on GCE). The servers are n1-highmem-4 (**4 vCPUs**, **26 GB memory**). On the server runs just mongod and there are no other services. **mongos** are on different servers. Usually process exit/crash happens once a day. If mongod process is killed by **oom-killer** this can be seen in the logs: Jun 15 14:45:17 server4 kernel: [1731430.432189] Out of memory: Kill process 13130 (mongod) score 980 or sacrifice child Jun 15 14:45:17 server4 kernel: [1731430.441717] Killed process 13130 (mongod) total-vm:28280536kB, anon-rss:26174876kB, file-rss:0kB, shmem-rss:0kB Sometimes mongod exits with leaving this in the mongod.log: 2018-06-15T02:14:32.456+0200 F - [rsSync] out of memory. 0x55cbc8535751 0x55cbc8534d84 0x55cbc8623b4b 0x55cbc86c665c 0x55cbc70fccff 0x55cbc70f8b02 0x55cbc707b3f1 0x55cbc86449b0 0x7fbbf3507494 0x7fbbf3249acf ----- BEGIN BACKTRACE ----- {"backtrace":[{"b":"55CBC6305000","o":"2230751","s":"_ZN5mongo15printStackTraceERSo"},{"b":"55CBC6305000","o":"222FD84","s":"_ZN5mongo29reportOutOfMemoryErrorAndExitEv"},{"b":"55CBC6305000","o":"231EB4B"},{"b":"55CBC6305000","o":"23C165C","s":"_Znam"},{"b":"55CBC6305000","o":"DF7CFF","s":"_ZN5mongo4repl8SyncTail7OpQueueC1Ev"},{"b":"55CBC6305000","o":"DF3B02","s":"_ZN5mongo4repl8SyncTail16oplogApplicationEPNS0_22ReplicationCoordinatorE"},{"b":"55CBC6305000","o":"D763F1","s":"_ZN5mongo4repl10RSDataSync4_runEv"},{"b":"55CBC6305000","o":"233F9B0"},{"b":"7FBBF3500000","o":"7494"},{"b":"7FBBF3161000","o":"E8ACF","s":"clone"}],"processInfo":{ "mongodbVersion" : "3.6.5", "gitVersion" : "a20ecd3e3a174162052ff99913bc2ca9a839d618", "compiledModules" : [], "uname" : { "sysname" : "Linux", "release" : "4.9.0-6-amd64", "version" : "#1 SMP Debian 4.9.88-1+deb9u1 (2018-05-07)", "machine" : "x86_64" }, "somap" : [ { "b" : "55CBC6305000", "elfType" : 3, "buildId" : "7D4592BDFAA6C15459D2319DEAB7F10E9EB4E7D7" }, { "b" : "7FFC48D98000", "path" : "linux-vdso.so.1", "elfType" : 3, "buildId" : "A3207CC9FE1CAA3374AE7061AA5C3C5619B8A0E5" }, { "b" : "7FBBF4743000", "path" : "/lib/x86_64-linux-gnu/libresolv.so.2", "elfType" : 3, "buildId" : "713D47D5F599289C0A91ADE8F0122B2B4AA78B2E" }, { "b" : "7FBBF42B0000", "path" : "/usr/lib/x86_64-linux-gnu/libcrypto.so.1.1", "elfType" : 3, "buildId" : "2CFE882A331D7857E9CE1B5DE3255E6DA76EF899" }, { "b" : "7FBBF4044000", "path" : "/usr/lib/x86_64-linux-gnu/libssl.so.1.1", "elfType" : 3, "buildId" : "E2AA3B39763D943F56B3BD05C8E36E639BA95E12" }, { "b" : "7FBBF3E40000", "path" : "/lib/x86_64-linux-gnu/libdl.so.2", "elfType" : 3, "buildId" : "B895F0831F623C5F23603401D4069F9F94C24761" }, { "b" : "7FBBF3C38000", "path" : "/lib/x86_64-linux-gnu/librt.so.1", "elfType" : 3, "buildId" : "5D83E0642E645026DBB11F89F7DF7106BD821495" }, { "b" : "7FBBF3934000", "path" : "/lib/x86_64-linux-gnu/libm.so.6", "elfType" : 3, "buildId" : "1B95E3A8B8788B07E4F59EE69B1877F9DEB42033" }, { "b" : "7FBBF371D000", "path" : "/lib/x86_64-linux-gnu/libgcc_s.so.1", "elfType" : 3, "buildId" : "51AD5FD294CD6C813BED40717347A53434B80B7A" }, { "b" : "7FBBF3500000", "path" : "/lib/x86_64-linux-gnu/libpthread.so.0", "elfType" : 3, "buildId" : "4285CD3158DDE596765C747AE210AB6CBD258B22" }, { "b" : "7FBBF3161000", "path" : "/lib/x86_64-linux-gnu/libc.so.6", "elfType" : 3, "buildId" : "AA889E26A70F98FA8D230D088F7CC5BF43573163" }, { "b" : "7FBBF495A000", "path" : "/lib64/ld-linux-x86-64.so.2", "elfType" : 3, "buildId" : "263F909DBE11A66F7C6233E3FF0521148D9F8370" } ] }} mongod(_ZN5mongo15printStackTraceERSo+0x41) [0x55cbc8535751] mongod(_ZN5mongo29reportOutOfMemoryErrorAndExitEv+0x84) [0x55cbc8534d84] mongod(+0x231EB4B) [0x55cbc8623b4b] mongod(_Znam+0x21C) [0x55cbc86c665c] mongod(_ZN5mongo4repl8SyncTail7OpQueueC1Ev+0x7F) [0x55cbc70fccff] mongod(_ZN5mongo4repl8SyncTail16oplogApplicationEPNS0_22ReplicationCoordinatorE+0x402) [0x55cbc70f8b02] mongod(_ZN5mongo4repl10RSDataSync4_runEv+0x111) [0x55cbc707b3f1] mongod(+0x233F9B0) [0x55cbc86449b0] libpthread.so.0(+0x7494) [0x7fbbf3507494] libc.so.6(clone+0x3F) [0x7fbbf3249acf] ----- END BACKTRACE ----- We have tried to play with the cacheSizeGB parameter and reduced it to 10GB: cacheSizeGB: 10.0 But still the crash happens. It is worth mentioning that there is a chunk moving process underway and that mongod instances that crash are the one from the shard that receive chunks. What tuning options to use to avoid such crashes? UPDATE: added small swap od 1GB, but still out-of-memory happens.

I am configuring a mongodb replica set. From my current primary node, when rs.add('host1:27017'), it yields this error

Quorum check failed because not enough voting nodes responded; required 2 but only the following 1 voting nodes responded; the following nodes did not respond affirmatively; failed with Server min and max wire version are incompatible (7,7) with client min wire version (6,6)

On my host1 and host2 machines, I already added replication option with the same replSetName, bind_ip, and exposed firewall. As a proof, through command line mongo --host host1 I can still connect to host1's mongo instance. The telnet command also yields the same successful connection. I don't know how to fix this.

I have a Mongo replica set on Ubuntu with three members configured for my information system and am having issues when I try to upgrade MongoDB from version 3.4 to 3.6 After setting the "featureCompatibilityVersion" to 3.4 on the master node and checking it on the two slaves, I started upgrading the slaves one by one (I gracefully shut the DB down with db.shutdownServer(), update the packages using apt and start the mongod service). That all went fine and the slaves transitioned back to secondary states. I then shut the master down, repeated all the steps for the upgrade from the official guide (https://docs.mongodb.com/manual/release-notes/3.6-upgrade-replica-set/) and tried starting the mongod service again. However, while I was upgrading the master node, the slave that was elected as a new master crashed with the error: >[rsSync] Fatal assertion 40454 DuplicateKey: E11000 duplicate key error collection: admin.system.users index: user_1_db_1 dup key: { : "john", : "admin" } at src/mongo/db/system_index.cpp 134 I didn't notice this and when the old master started up again it too crashed with the same error. In my frantic trying to get the members up again, the third member had also been elected as a master for a short time before crashing with the same error. When I managed to get the first member up again as a standalone, I checked the system.users collection and found that there were indeed two users with the same username for the admin database (I redacted the parts with ...):

{ "_id" : ObjectId("5...8"), "user" : "john", "db" : "admin", "credentials" : { "MONGODB-CR" : "f...c" }, "roles" : [ { "role" : "read", "db" : "admin" } ] }

and

{ "_id" : "admin.john", "user" : "john", "db" : "admin", "credentials" : { "MONGODB-CR" : "f...c" }, "roles" : [ { "role" : "read", "db" : "admin" } ] }

My question is how is that possible? Shouldn't mongo prevent someone from adding an identical user? Or did mongo try to change that collection after the upgrade and failed? I am using authSchema version 3 if it makes any difference. Thanks!

For some reasons I disabled the user access control and somehow all collections have been wiped out. So I want an activity log to find out how this happened I need a list like this:

|  Date/Time           |  IP           |  Command           |
|----------------------+---------------+--------------------|
|  2018/1/1  18:20:50  |  127.0.0.1    |  show collections  |
|  yyyy/m/d  hh:mm:ss  |  x.x.x.x      |  executed command  |

* OS: **Ubuntu 18.04** * MongoDB: **3.6.3**