Mongoose: Schemas
Mongoose: Schemas
Mongoose: Schemas
Schemas
If you haven't yet done so, please take a minute to read the quickstart to get an idea of how
Mongoose works. If you are migrating from 5.x to 6.x please take a moment to read the migration
guide.
Everything in Mongoose starts with a Schema. Each schema maps to a MongoDB collection and
de nes the shape of the documents within that collection.
If you want to add additional keys later, use the Schema#add method.
Each key in our code blogSchema de nes a property in our documents which will be cast to its
associated SchemaType. For example, we've de ned a property title which will be cast to the
String SchemaType and property date which will be cast to a Date SchemaType.
Notice above that if a property only requires a type, it can be speci ed using a shorthand
notation (contrast the title property above with the date property).
Keys may also be assigned nested objects containing further key/type de nitions like the meta
property above. This will happen whenever a key's value is a POJO that doesn't have a type
property.
In these cases, Mongoose only creates actual schema paths for leaves in the tree. (like
meta.votes and meta.favs above), and the branches do not have actual paths. A side-e ect of
this is that meta above cannot have its own validation. If validation is needed up the tree, a path
needs to be created up the tree - see the Subdocuments section for more information on how to
do this. Also read the Mixed subsection of the SchemaTypes guide for some gotchas.
String
Number
Date
Bu er
Boolean
Mixed
ObjectId
Array
Decimal128
Map
Schemas not only de ne the structure of your document and casting of properties, they also
de ne document instance methods, static Model methods, compound indexes, and document
lifecycle hooks called middleware.
Creating a model
To use our schema de nition, we need to convert our blogSchema into a Model we can work
with. To do so, we pass it into mongoose.model(modelName, schema) :
Ids
When you create a new document with the automatically added _id property, Mongoose
creates a new _id of type ObjectId to your document.
You can also overwrite Mongoose's default _id with your own _id . Just be careful: Mongoose
will refuse to save a document that doesn't have an _id , so you're responsible for setting _id if
you de ne your own _id path.
doc._id = 1;
await doc.save(); // works
Instance methods
Instances of Models are documents. Documents have many of their own built-in instance
methods. We may also de ne our own custom document instance methods.
// define a schema
const animalSchema = new Schema({ name: String, type: String },
{
// Assign a function to the "methods" object of our animalSchema through schema options.
// By following this approach, there is no need to create a separate TS type to define the
methods: {
findSimilarTypes(cb) {
return mongoose.model('Animal').find({ type: this.type }, cb);
}
}
});
Now all of our animal instances have a findSimilarTypes method available to them.
Overwriting a default mongoose document method may lead to unpredictable results. See
this for more details.
The example above uses the Schema.methods object directly to save an instance method.
You can also use the Schema.method() helper as described here.
Do not declare methods using ES6 arrow functions ( => ). Arrow functions explicitly prevent
binding this , so your method will not have access to the document and the above
examples will not work.
Statics
You can also add static functions to your model. There are three equivalent ways to add a static:
// define a schema
const animalSchema = new Schema({ name: String, type: String },
{
// Assign a function to the "statics" object of our animalSchema through schema options.
// By following this approach, there is no need to create a separate TS type to define the
statics: {
findByName(name) {
return this.find({ name: new RegExp(name, 'i') });
}
}
}); check all these three syntaxes
Do not declare statics using ES6 arrow functions ( => ). Arrow functions explicitly prevent binding
this , so the above examples will not work because of the value of this .
Query Helpers
You can also add query helper functions, which are like instance methods but for mongoose
queries. Query helper methods let you extend mongoose's chainable query builder API.
// define a schema
const animalSchema = new Schema({ name: String, type: String },
{
// Assign a function to the "query" object of our animalSchema through schema options.
// By following this approach, there is no need to create a separate TS type to define the
query:{
byName(name){
return this.where({ name: new RegExp(name, 'i') })
}
}
});
Indexes
MongoDB supports secondary indexes. With mongoose, we de ne these indexes within our
Schema at the path level or the schema level. De ning indexes at the schema level is necessary
when creating compound indexes.
When your application starts up, Mongoose automatically calls createIndex for each de ned
index in your schema. Mongoose will call createIndex for each index sequentially, and emit an
'index' event on the model when all the createIndex calls succeeded or when there was an
error. While nice for development, it is recommended this behavior be disabled in production
since index creation can cause a signi cant performance impact. Disable the behavior by setting
the autoIndex option of your schema to false , or globally on the connection by setting the
option autoIndex to false .
Mongoose will emit an index event on the model when indexes are done building or an error
occurred.
// Will cause an error because mongodb has an _id index by default that
// is not sparse
animalSchema.index({ _id: 1 }, { sparse: true });
const Animal = mongoose.model('Animal', animalSchema);
Virtuals
Virtuals are document properties that you can get and set but that do not get persisted to
MongoDB. The getters are useful for formatting or combining elds, while setters are useful for
de-composing a single value into multiple values for storage.
// define a schema
const personSchema = new Schema({
name: {
first: String,
last: String
}
});
// create a document
const axl = new Person({
name: { first: 'Axl', last: 'Rose' }
});
Suppose you want to print out the person's full name. You could do it yourself:
But concatenating the rst and last name every time can get cumbersome. And what if you want
to do some extra processing on the name, like removing diacritics? A virtual property getter lets
you de ne a fullName property that won't get persisted to MongoDB.
Now, mongoose will call your getter function every time you access the fullName property:
If you use toJSON() or toObject() mongoose will not include virtuals by default. This includes
the output of calling JSON.stringify() on a Mongoose document, because JSON.stringify()
calls toJSON() . Pass { virtuals: true } to either toObject() or toJSON() .
You can also add a custom setter to your virtual that will let you set both rst name and last
name via the fullName virtual.
Virtual property setters are applied before other validation. So the example above would still
work even if the first and last name elds were required.
Only non-virtual properties work as part of queries and for eld selection. Since virtuals are not
stored in MongoDB, you can't query with them.
Aliases
Aliases are a particular type of virtual where the getter and setter seamlessly get and set another
property. This is handy for saving network bandwidth, so you can convert a short property name
stored in the database into a longer name for code readability.
You can also declare aliases on nested paths. It is easier to use nested schemas and
subdocuments, but you can also declare nested path aliases inline as long as you use the full
nested path nested.myProp as the alias.
Options
Schemas have a few con gurable options which can be passed to the constructor or to the set
method:
// or
Valid options:
autoIndex
autoCreate
bu erCommands
bu erTimeoutMS
capped
collection
discriminatorKey
id
_id
minimize
read
writeConcern
shardKey
statics
strict
strictQuery
toJSON
toObject
typeKey
validateBeforeSave
versionKey
optimisticConcurrency
collation
timeseries
selectPopulatedPaths
skipVersioning
timestamps
storeSubdocValidationError
methods
query
option: autoIndex
By default, Mongoose's init() function creates all the indexes de ned in your model's schema
by calling Model.createIndexes() after you successfully connect to MongoDB. Creating indexes
automatically is great for development and test environments. But index builds can also create
signi cant load on your production database. If you want to manage indexes carefully in
production, you can set autoIndex to false.
The autoIndex option is set to true by default. You can change this default by setting
mongoose.set('autoIndex', false);
option: autoCreate
option: bu erCommands
By default, mongoose bu ers commands when the connection goes down until the driver
manages to reconnect. To disable bu ering, set bufferCommands to false.
mongoose.set('bufferCommands', true);
// Schema option below overrides the above, if the schema option is set.
const schema = new Schema({..}, { bufferCommands: false });
option: bu erTimeoutMS
If bufferCommands is on, this option sets the maximum amount of time Mongoose bu ering will
wait before throwing an error. If not speci ed, Mongoose will use 10000 (10 seconds).
option: capped
The capped option may also be set to an object if you want to pass additional options like max
or autoIndexId. In this case you must explicitly pass the size option, which is required.
new Schema({..}, { capped: { size: 1024, max: 1000, autoIndexId: true } });
option: collection
Mongoose by default produces a collection name by passing the model name to the
utils.toCollectionName method. This method pluralizes the name. Set this option if you need a
di erent name for your collection. so if ur model name is test, ur collection will be tests
option: discriminatorKey
When you de ne a discriminator, Mongoose adds a path to your schema that stores which
discriminator a document is an instance of. By default, Mongoose adds an __t path, but you can
set discriminatorKey to overwrite this default.
option: id
Mongoose assigns each of your schemas an id virtual getter by default which returns the
document's _id eld cast to a string, or in the case of ObjectIds, its hexString. If you don't want
an id getter added to your schema, you may disable it by passing this option at schema
construction time.
// default behavior
const schema = new Schema({ name: String });
const Page = mongoose.model('Page', schema);
const p = new Page({ name: 'mongodb.org' });
console.log(p.id); // '50341373e894ad16347efe01'
// disabled id
const schema = new Schema({ name: String }, { id: false });
const Page = mongoose.model('Page', schema);
const p = new Page({ name: 'mongodb.org' });
console.log(p.id); // undefined
option: _id
Mongoose assigns each of your schemas an _id eld by default if one is not passed into the
Schema constructor. The type assigned is an ObjectId to coincide with MongoDB's default
behavior. If you don't want an _id added to your schema at all, you may disable it using this
option.
You can only use this option on subdocuments. Mongoose can't save a document without
knowing its id, so you will get an error if you try to save a document without an _id .
// default behavior
const schema = new Schema({ name: String });
const Page = mongoose.model('Page', schema);
const p = new Page({ name: 'mongodb.org' });
console.log(p); // { _id: '50341373e894ad16347efe01', name: 'mongodb.org' }
// disabled _id
const childSchema = new Schema({ name: String }, { _id: false });
const parentSchema = new Schema({ children: [childSchema] });
This behavior can be overridden by setting minimize option to false . It will then store empty
objects.
const schema = new Schema({ name: String, inventory: {} }, { minimize: false });
const Character = mongoose.model('Character', schema);
To check whether an object is empty, you can use the $isEmpty() helper:
sam.inventory.barrowBlade = 1;
sam.$isEmpty('inventory'); // false
option: read
Allows setting query#read options at the schema level, providing us a way to apply default
ReadPreferences to all queries derived from a model.
const schema = new Schema({..}, { read: 'primary' }); // also aliased as 'p'
const schema = new Schema({..}, { read: 'primaryPreferred' }); // aliased as 'pp'
const schema = new Schema({..}, { read: 'secondary' }); // aliased as 's'
const schema = new Schema({..}, { read: 'secondaryPreferred' }); // aliased as 'sp'
const schema = new Schema({..}, { read: 'nearest' }); // aliased as 'n'
The alias of each pref is also permitted so instead of having to type out 'secondaryPreferred' and
getting the spelling wrong, we can simply pass 'sp'.
The read option also allows us to specify tag sets. These tell the driver from which members of
the replica-set it should attempt to read. Read more about tag sets here and here.
NOTE: you may also specify the driver read preference strategy option when connecting:
option: writeConcern
option: shardKey
The shardKey option is used when we have a sharded MongoDB architecture. Each sharded
collection is given a shard key which must be present in all insert/update operations. We just
need to set this schema option to the same shard key and we’ll be all set.
Note that Mongoose does not send the shardcollection command for you. You must con gure
your shards yourself.
option: strict
The strict option, (enabled by default), ensures that values passed to our model constructor that
were not speci ed in our schema do not get saved to the db.
// set to false..
const thingSchema = new Schema({..}, { strict: false });
const thing = new Thing({ iAmNotInTheSchema: true });
thing.save(); // iAmNotInTheSchema is now saved to the db!!
This value can be overridden at the model instance level by passing a second boolean argument:
The strict option may also be set to "throw" which will cause errors to be produced instead
of dropping the bad data.
NOTE: Any key/val set on the instance that does not exist in your schema is always ignored,
regardless of schema option.
option: strictQuery
Mongoose supports a separate strictQuery option to avoid strict mode for query lters. This is
because empty query lters cause Mongoose to return all documents in the model, which can
cause issues.
The strict option does apply to updates. The strictQuery option is just for query lters.
Mongoose has a separate strictQuery option to toggle strict mode for the filter parameter
to queries.
// Don't do this!
const docs = await MyModel.find(req.query);
// Do this instead:
const docs = await MyModel.find({ name: req.query.name, age: req.query.age }).setOptions({ s
In Mongoose 6, strictQuery is equal to strict by default. However, you can override this
behavior globally:
In Mongoose 7, strictQuery default value will be switched back to false . You can prepare for
the change by specifying:
option: toJSON
Exactly the same as the toObject option but only applies when the document's toJSON method
is called.
option: toObject
Documents have a toObject method which converts the mongoose document into a plain
JavaScript object. This method accepts a few options. Instead of applying these options on a per-
document basis, we may declare the options at the schema level and have them applied to all of
the schema's documents by default.
To have all virtuals show up in your console.log output, set the toObject option to {
getters: true } :
option: typeKey
By default, if you have an object with key 'type' in your schema, mongoose will interpret it as a
type declaration.
However, for applications like geoJSON, the 'type' property is important. If you want to control
which key mongoose uses to nd type declarations, set the 'typeKey' schema option.
option: validateBeforeSave
By default, documents are automatically validated before they are saved to the database. This is
to prevent saving an invalid document. If you want to handle validation manually, and be able to
save objects which don't pass validation, you can set validateBeforeSave to false.
option: versionKey
The versionKey is a property set on each document when rst created by Mongoose. This keys
value contains the internal revision of the document. The versionKey option is a string that
represents the path to use for versioning. The default is __v . If this con icts with your
application you can con gure as such:
// customized versionKey
new Schema({..}, { versionKey: '_somethingElse' })
const Thing = mongoose.model('Thing', schema);
const thing = new Thing({ name: 'mongoose v3' });
thing.save(); // { _somethingElse: 0, name: 'mongoose v3' }
Note that Mongoose's default versioning is not a full optimistic concurrency solution. Mongoose's
default versioning only operates on arrays as shown below.
If you need optimistic concurrency support for save() , you can set the optimisticConcurrency
option
Document versioning can also be disabled by setting the versionKey to false . DO NOT disable
versioning unless you know what you are doing.
Mongoose only updates the version key when you use save() . If you use update() ,
findOneAndUpdate() , etc. Mongoose will not update the version key. As a workaround, you can
use the below middleware.
schema.pre('findOneAndUpdate', function() {
const update = this.getUpdate();
if (update.__v != null) {
delete update.__v;
}
const keys = ['$set', '$setOnInsert'];
for (const key of keys) {
if (update[key] != null && update[key].__v != null) {
delete update[key].__v;
if (Object.keys(update[key]).length === 0) {
delete update[key];
}
}
}
update.$inc = update.$inc || {};
update.$inc.__v = 1;
});
option: optimisticConcurrency
Optimistic concurrency is a strategy to ensure the document you're updating didn't change
between when you loaded it using find() or findOne() , and when you update it using
save() .
For example, suppose you have a House model that contains a list of photos , and a status
that represents whether this house shows up in searches. Suppose that a house that has status
'APPROVED' must have at least two photos . You might implement the logic of approving a
house document as shown below:
house.status = 'APPROVED';
await house.save();
}
The markApproved() function looks right in isolation, but there might be a potential issue: what
if another function removes the house's photos between the findOne() call and the save()
call? For example, the below code will succeed:
If you set the optimisticConcurrency option on the House model's schema, the above script
will throw an error.
option: collation
Sets a default collation for every query and aggregation. Here's a beginner-friendly overview of
collations.
option: timeseries
If you set the timeseries option on a schema, Mongoose will create a timeseries collection for
any model that you create from that schema.
option: skipVersioning
skipVersioning allows excluding paths from versioning (i.e., the internal revision will not be
incremented even if these paths are updated). DO NOT do this unless you know what you're
doing. For subdocuments, include this on the parent document using the fully quali ed path.
option: timestamps
The timestamps option tells Mongoose to assign createdAt and updatedAt elds to your
schema. The type assigned is Date.
By default, the names of the elds are createdAt and updatedAt . Customize the eld names by
setting timestamps.createdAt and timestamps.updatedAt .
If you create a new document, mongoose simply sets createdAt , and updatedAt to the
time of creation.
If you update a document, mongoose will add updatedAt to the $set object.
If you set upsert: true on an update operation, mongoose will use $setOnInsert
operator to add createdAt to the document in case the upsert operation resulted into a
new inserted document.
// If you set upsert: true, Mongoose will add `created_at` to `$setOnInsert` as well
await Thing.findOneAndUpdate({}, { $set: { name: 'Test2' } });
By default, Mongoose uses new Date() to get the current time. If you want to overwrite the
function Mongoose uses to get the current time, you can set the timestamps.currentTime
option. Mongoose will call the timestamps.currentTime function whenever it needs to get the
current time.
option: pluginTags
Mongoose supports de ning global plugins, plugins that apply to all schemas.
Sometimes, you may only want to apply a given plugin to some schemas. In that case, you can
add pluginTags to a schema:
If you call plugin() with a tags option, Mongoose will only apply that plugin to schemas that
have a matching entry in pluginTags .
option: selectPopulatedPaths
By default, Mongoose will automatically select() any populated paths for you, unless you
explicitly exclude them.
To opt out of selecting populated elds by default, set selectPopulatedPaths to false in your
schema.
option: storeSubdocValidationError
For legacy reasons, when there is a validation error in subpath of a single nested schema,
Mongoose will record that there was a validation error in the single nested schema path as well.
For example:
const childSchema = new Schema({ name: { type: String, required: true } });
const parentSchema = new Schema({ child: childSchema });
Set the storeSubdocValidationError to false on the child schema to make Mongoose only
reports the parent error.
Schemas have a loadClass() method that you can use to create a Mongoose schema from an
ES6 class:
class MyClass {
myMethod() { return 42; }
static myStatic() { return 42; }
get myVirtual() { return 42; }
}
Pluggable
Schemas are also pluggable which allows us to package up reusable features into plugins that
can be shared with the community or just between your projects.
Further Reading
To get the most out of MongoDB, you need to learn the basics of MongoDB schema design. SQL
schema design (third normal form) was designed to minimize storage costs, whereas MongoDB
schema design is about making common queries as fast as possible. The 6 Rules of Thumb for
MongoDB Schema Design blog series is an excellent resource for learning the basic rules for
making your queries fast.
Users looking to master MongoDB schema design in Node.js should look into The Little MongoDB
Schema Design Book by Christian Kvalheim, the original author of the MongoDB Node.js driver.
This book shows you how to implement performant schemas for a laundry list of use cases,
including e-commerce, wikis, and appointment bookings.
Next Up
SchemaTypes
SchemaTypes handle de nition of path defaults, validation, getters, setters, eld selection
defaults for queries, and other general characteristics for Mongoose document properties.
What is a SchemaType?
The type Key
SchemaType Options
Usage Notes
Getters
Custom Types
The schema.path() Function
Further Reading
What is a SchemaType?
You can think of a Mongoose schema as the con guration object for a Mongoose model. A
SchemaType is then a con guration object for an individual property. A SchemaType says what
type a given path should have, whether it has any getters/setters, and what values are valid for
that path.
The following are all the valid SchemaTypes in Mongoose. Mongoose plugins can also add
custom SchemaTypes like int32. Check out Mongoose's plugins search to nd plugins.
String
Number
Date
Bu er
Boolean
Mixed
ObjectId
Array
Decimal128
Map
Schema
Example
// example use
type is a special property in Mongoose schemas. When Mongoose nds a nested property
named type in your schema, Mongoose assumes that it needs to de ne a SchemaType with the
given type.
As a consequence, you need a little extra work to de ne a property named type in your schema.
For example, suppose you're building a stock portfolio app, and you want to store the asset's
type (stock, bond, ETF, etc.). Naively, you might de ne your schema as shown below:
However, when Mongoose sees type: String , it assumes that you mean asset should be a
string, not an object with a property type . The correct way to de ne an object with a property
type is shown below.
SchemaType Options
You can declare a schema type using the type directly, or an object with a type property.
In addition to the type property, you can specify additional properties for a path. For example, if
you want to lowercase a string before saving:
You can add any property you want to your SchemaType options. Many plugins rely on custom
SchemaType options. For example, the mongoose-autopopulate plugin automatically populates
paths if you set autopopulate: true in your SchemaType options. Mongoose comes with
support for several built-in SchemaType options, like lowercase in the above example.
The lowercase option only works for strings. There are certain options which apply for all
schema types, and some that apply for speci c schema types.
required : boolean or function, if true adds a required validator for this property
default : Any or function, sets a default value for the path. If the value is a function, the
return value of the function is used as the default.
select : boolean, speci es default projections for queries
validate : function, adds a validator function for this property
get : function, de nes a custom getter for this property using Object.defineProperty() .
set : function, de nes a custom setter for this property using Object.defineProperty() .
alias : string, mongoose >= 4.10.0 only. De nes a virtual with the given name that
gets/sets this path.
immutable : boolean, de nes path as immutable. Mongoose prevents you from changing
immutable paths unless the parent document has isNew: true .
transform : function, Mongoose calls this function when you call Document#toJSON()
function, including when you JSON.stringify() a document.
Indexes
String
Number
min : Number, creates a validator that checks if the value is greater than or equal to the
given minimum.
max : Number, creates a validator that checks if the value is less than or equal to the given
maximum.
enum : Array, creates a validator that checks if the value is strictly equal to one of the values
in the given array.
populate : Object, sets default populate options
Date
min : Date, creates a validator that checks if the value is greater than or equal to the given
minimum.
max : Date, creates a validator that checks if the value is less than or equal to the given
maximum.
expires : Number or String, creates a TTL index with the value expressed in seconds.
ObjectId
Usage Notes
String
To declare a path as a string, you may use either the String global constructor or the string
'String' .
const schema1 = new Schema({ name: String }); // name will be cast to string
const schema2 = new Schema({ name: 'String' }); // Equivalent
If you pass an element that has a toString() function, Mongoose will call it, unless the element
is an array or the toString() function is strictly equal to Object.prototype.toString() .
Number
To declare a path as a number, you may use either the Number global constructor or the string
'Number' .
const schema1 = new Schema({ age: Number }); // age will be cast to a Number
const schema2 = new Schema({ age: 'Number' }); // Equivalent
There are several types of values that will be successfully cast to a Number.
If you pass an object with a valueOf() function that returns a Number, Mongoose will call it and
assign the returned value to the path.
NaN, strings that cast to NaN, arrays, and objects that don't have a valueOf() function will all
result in a CastError once validated, meaning that it will not throw on initialization, only when
validated. validation is done while saving in the database
Dates
Built-in Date methods are not hooked into the mongoose change tracking logic which in English
means that if you use a Date in your document and modify it with a method like setMonth() ,
mongoose will be unaware of this change and doc.save() will not persist this modi cation. If
you must modify Date types using built-in methods, tell mongoose about the change with
doc.markModified('pathToYourDate') before saving.
doc.markModified('dueDate');
doc.save(callback); // works
})
Bu er
To declare a path as a Bu er, you may use either the Buffer global constructor or the string
'Buffer' .
const schema1 = new Schema({ binData: Buffer }); // binData will be cast to a Buffer
const schema2 = new Schema({ binData: 'Buffer' }); // Equivalent
Mixed
An "anything goes" SchemaType. Mongoose will not do any casting on mixed paths. You can
de ne a mixed path using Schema.Types.Mixed or by passing an empty object literal. The
following are equivalent.
Since Mixed is a schema-less type, you can change the value to anything else you like, but
Mongoose loses the ability to auto detect and save those changes. To tell Mongoose that the
value of a Mixed type has changed, you need to call doc.markModified(path) , passing the path
to the Mixed type you just changed.
ObjectIds
An ObjectId is a special type typically used for unique identi ers. Here's how you declare a
schema with a path driver that is an ObjectId:
ObjectId is a class, and ObjectIds are objects. However, they are often represented as strings.
When you convert an ObjectId to a string using toString() , you get a 24-character hexadecimal
string:
Boolean
Booleans in Mongoose are plain JavaScript booleans. By default, Mongoose casts the below
values to true :
true
'true'
1
'1'
'yes'
false
'false'
0
'0'
'no'
Any other value causes a CastError. You can modify what values Mongoose converts to true or
false using the convertToTrue and convertToFalse properties, which are JavaScript sets.
mongoose.Schema.Types.Boolean.convertToFalse.add('nay');
console.log(new M({ b: 'nay' }).b); // false
Arrays
Arrays are special because they implicitly have a default value of [] (empty array).
To overwrite this default, you need to set the default value to undefined
Note: specifying an empty array is equivalent to Mixed . The following all create arrays of Mixed :
Maps
A MongooseMap is a subclass of JavaScript's Map class. In these docs, we'll use the terms 'map'
and MongooseMap interchangeably. In Mongoose, maps are how you create a nested document
with arbitrary keys.
Note: In Mongoose Maps, keys must be strings in order to store the document in MongoDB.
The above example doesn't explicitly declare github or twitter as paths, but, since
socialMediaHandles is a map, you can store arbitrary key/value pairs. However, since
socialMediaHandles is a map, you must use .get() to get the value of a key and .set() to
set the value of a key.
// Good
user.socialMediaHandles.set('github', 'vkarpov15');
// Works too
user.set('socialMediaHandles.twitter', '@code_barbarian');
// Bad, the `myspace` property will **not** get saved
user.socialMediaHandles.myspace = 'fail';
// 'vkarpov15'
console.log(user.socialMediaHandles.get('github'));
// '@code_barbarian'
console.log(user.get('socialMediaHandles.twitter'));
// undefined
user.socialMediaHandles.github;
Map types are stored as BSON objects in MongoDB. Keys in a BSON object are ordered, so this
means the insertion order property of maps is maintained.
Mongoose supports a special $* syntax to populate all elements in a map. For example, suppose
your socialMediaHandles map contains a ref :
Getters
Getters are like virtuals for paths de ned in your schema. For example, let's say you wanted to
store user pro le pictures as relative paths and then add the hostname in your application.
Below is how you would structure your userSchema :
Generally, you only use getters on primitive paths as opposed to arrays or subdocuments.
Because getters override what accessing a Mongoose path returns, declaring a getter on an
object may remove Mongoose change tracking for that path.
// Later
doc.arr.push({ key: String });
doc.arr[0]; // 'undefined' because every `doc.arr` creates a new array!
Instead of declaring a getter on the array as shown above, you should declare a getter on the
url string as shown below. If you need to declare a getter on a nested document or array, be
very careful!
Schemas
To set a default value based on the sub-schema's shape, simply set a default value, and the value
will be cast based on the sub-schema's de nition before being set during document creation.
Mongoose can also be extended with custom SchemaTypes. Search the plugins site for
compatible types like mongoose-long, mongoose-int32, and other types.
The schema.path() function returns the instantiated schema type for a given path.
const sampleSchema = new Schema({ name: { type: String, required: true } });
console.log(sampleSchema.path('name'));
// Output looks like:
/**
* SchemaString {
* enumValues: [],
* regExp: null,
* path: 'name',
* instance: 'String',
* validators: ...
*/
You can use this function to inspect the schema type for a given path, including what validators it
has and what the type is.
Further Reading
Next Up
Connections
mongoose.connect('mongodb://127.0.0.1:27017/myapp');
This is the minimum needed to connect the myapp database running locally on the default port
(27017). If connecting fails on your machine, try using 127.0.0.1 instead of localhost .
mongoose.connect('mongodb://username:password@host:port/database?options...');
Bu ering
Error Handling
Options
Connection String Options
Connection Events
A note about keepAlive
Server Selection
Replica Set Connections
Replica Set Host Names
Multi-mongos support
Multiple connections
Connection Pools
Operation Bu ering
Mongoose lets you start using your models immediately, without waiting for mongoose to
establish a connection to MongoDB.
mongoose.connect('mongodb://127.0.0.1:27017/myapp');
const MyModel = mongoose.model('Test', new Schema({ name: String }));
// Works
MyModel.findOne(function(error, result) { /* ... */ });
That's because mongoose bu ers model function calls internally. This bu ering is convenient,
but also a common source of confusion. Mongoose will not throw any errors by default if you use
a model without connecting.
setTimeout(function() {
mongoose.connect('mongodb://127.0.0.1:27017/myapp');
}, 60000);
To disable bu ering, turn o the bufferCommands option on your schema. If you have
bufferCommands on and your connection is hanging, try turning bufferCommands o to see if
you haven't opened a connection properly. You can also disable bufferCommands globally:
mongoose.set('bufferCommands', false);
Note that bu ering is also responsible for waiting until Mongoose creates collections if you use
the autoCreate option. If you disable bu ering, you should also disable the autoCreate option
and use createCollection() to create capped collections or collections with collations.
Error Handling
There are two classes of errors that can occur with a Mongoose connection.
Error on initial connection. If initial connection fails, Mongoose will emit an 'error' event and
the promise mongoose.connect() returns will reject. However, Mongoose will not
automatically try to reconnect.
Error after initial connection was established. Mongoose will attempt to reconnect, and it
will emit an 'error' event.
To handle initial connection errors, you should use .catch() or try/catch with async/await.
error is thrown by this promise
mongoose.connect('mongodb://127.0.0.1:27017/test').
catch(error => handleError(error));
// Or:
try {
await mongoose.connect('mongodb://127.0.0.1:27017/test');
} catch (error) {
handleError(error);
}
To handle errors after initial connection was established, you should listen for error events on
the connection. However, you still need to handle initial connection errors as shown above.
Note that Mongoose does not necessarily emit an 'error' event if it loses connectivity to
MongoDB. You should listen to the disconnected event to report when Mongoose is
disconnected from MongoDB.
Options
The connect method also accepts an options object which will be passed on to the underlying
MongoDB driver.
mongoose.connect(uri, options);
A full list of options can be found on the MongoDB Node.js driver docs for MongoClientOptions .
Mongoose passes options to the driver without modi cation, modulo a few exceptions that are
explained below.
Below are some of the options that are important for tuning Mongoose.
The serverSelectionTimeoutMS option also handles how long mongoose.connect() will retry
initial connection before erroring out. mongoose.connect() will retry for 30 seconds by default
(default serverSelectionTimeoutMS ) before erroring out. To get faster feedback on failed
operations, you can reduce serverSelectionTimeoutMS to 5000 as shown below.
Example:
const options = {
autoIndex: false, // Don't build indexes
maxPoolSize: 10, // Maintain up to 10 socket connections
serverSelectionTimeoutMS: 5000, // Keep trying to send operations for 5 seconds
socketTimeoutMS: 45000, // Close sockets after 45 seconds of inactivity
family: 4 // Use IPv4, skip trying IPv6
};
mongoose.connect(uri, options);
See this page for more information about connectTimeoutMS and socketTimeoutMS
Callback
The connect() function also accepts a callback parameter and returns a promise.
// Or using promises
mongoose.connect(uri, options).then(
() => { /** ready to use. The `mongoose.connect()` promise resolves to mongoose instance.
err => { /** handle initial connection error */ }
);
You can also specify driver options in your connection string as parameters in the query string
portion of the URI. This only applies to options passed to the MongoDB driver. You can't set
Mongoose-speci c options like bufferCommands in the query string.
mongoose.connect('mongodb://127.0.0.1:27017/test?connectTimeoutMS=1000&bufferCommands=false&
// The above is equivalent to:
mongoose.connect('mongodb://127.0.0.1:27017/test', {
connectTimeoutMS: 1000
// Note that mongoose will **not** pull `bufferCommands` from the query string
});
The disadvantage of putting options in the query string is that query string options are harder to
read. The advantage is that you only need a single con guration option, the URI, rather than
separate options for socketTimeoutMS , connectTimeoutMS , etc. Best practice is to put options
that likely di er between development and production, like replicaSet or ssl , in the
connection string, and options that should remain constant, like connectTimeoutMS or
maxPoolSize , in the options object.
The MongoDB docs have a full list of supported connection string options. Below are some
options that are often useful to set in the connection string because they are closely associated
with the hostname and authentication information.
authSource - The database to use when authenticating with user and pass . In MongoDB,
users are scoped to a database. If you are getting an unexpected login failure, you may
need to set this option.
family - Whether to connect using IPv4 or IPv6. This option passed to Node.js'
dns.lookup() function. If you don't specify this option, the MongoDB driver will try IPv6
rst and then IPv4 if IPv6 fails. If your mongoose.connect(uri) call takes a long time, try
mongoose.connect(uri, { family: 4 })
Connection Events
Connections inherit from Node.js' EventEmitter class, and emit events when something
happens to the connection, like losing connectivity to the MongoDB server. Below is a list of
events that a connection may emit.
connecting : Emitted when Mongoose starts making its initial connection to the MongoDB
server
connected : Emitted when Mongoose successfully makes its initial connection to the
MongoDB server, or when Mongoose reconnects after losing connectivity. May be emitted
multiple times if Mongoose loses connectivity.
open : Emitted after 'connected' and onOpen is executed on all of this connection's
models.
disconnecting : Your app called Connection#close() to disconnect from MongoDB
disconnected : Emitted when Mongoose lost connection to the MongoDB server. This
event may be due to your code explicitly closing the connection, the database server
crashing, or network connectivity issues.
close : Emitted after Connection#close() successfully closes the connection. If you call
conn.close() , you'll get both a 'disconnected' event and a 'close' event.
reconnected : Emitted if Mongoose lost connectivity to MongoDB and successfully
reconnected. Mongoose attempts to automatically reconnect when it loses connection to
the database.
error : Emitted if an error occurs on a connection, like a parseError due to malformed
data or a payload larger than 16MB.
fullsetup : Emitted when you're connecting to a replica set and Mongoose has
successfully connected to the primary and at least one secondary.
all : Emitted when you're connecting to a replica set and Mongoose has successfully
connected to all servers speci ed in your connection string.
When you're connecting to a single MongoDB server (a "standalone"), Mongoose will emit
'disconnected' if it gets disconnected from the standalone server, and 'connected' if it
successfully connects to the standalone. In a replica set, Mongoose will emit 'disconnected' if it
loses connectivity to the replica set primary, and 'connected' if it manages to reconnect to the
replica set primary.
For long running applications, it is often prudent to enable keepAlive with a number of
milliseconds. Without it, after some period of time you may start to see "connection closed"
errors for what seems like no reason. If so, after reading this, you may decide to enable
keepAlive :
To connect to a replica set you pass a comma delimited list of hosts to connect to rather than a
single host.
mongoose.connect('mongodb://[username:password@]host1[:port1][,host2[:port2],...[,hostN[:por
For example:
mongoose.connect('mongodb://user:[email protected]:27017,host2.com:27017,host3.com:27017/testdb')
mongoose.connect('mongodb://host1:port1/?replicaSet=rsName');
Server Selection
The underlying MongoDB driver uses a process known as server selection to connect to
MongoDB and send operations to MongoDB. If the MongoDB driver can't nd a server to send an
operation to after serverSelectionTimeoutMS , you'll get the below error:
You can con gure the timeout using the serverSelectionTimeoutMS option to
mongoose.connect() :
mongoose.connect(uri, {
serverSelectionTimeoutMS: 5000 // Timeout after 5s instead of 30s
});
A MongoTimeoutError has a reason property that explains why server selection timed out. For
example, if you're connecting to a standalone server with an incorrect password, reason will
contain an "Authentication failed" error.
MongoDB replica sets rely on being able to reliably gure out the domain name for each
member. On Linux and OSX, the MongoDB server uses the output of the hostname command to
gure out the domain name to report to the replica set. This can cause confusing errors if you're
connecting to a remote MongoDB replica set running on a machine that reports its hostname as
localhost :
// Can get this error even if your connection string doesn't include
// `localhost` if `rs.conf()` reports that one replica set member has
// `localhost` as its host name.
MongooseServerSelectionError: connect ECONNREFUSED localhost:27017
If you're experiencing a similar error, connect to the replica set using the mongo shell and run the
rs.conf() command to check the host names of each replica set member. Follow this page's
instructions to change a replica set member's host name.
Multi-mongos support
You can also connect to multiple mongos instances for high availability in a sharded cluster. You
do not need to pass any special options to connect to multiple mongos in mongoose 5.x.
Multiple connections
So far we've seen how to connect to MongoDB using Mongoose's default connection. Mongoose
creates a default connection when you call mongoose.connect() . You can access the default
connection using mongoose.connection .
You may need multiple connections to MongoDB for several reasons. One reason is if you have
multiple databases or multiple MongoDB clusters. Another reason is to work around slow trains.
The mongoose.createConnection() function takes the same arguments as
mongoose.connect() and returns a new connection.
This connection object is then used to create and retrieve models. Models are always scoped to a
single connection.
If you use multiple connections, you should make sure you export schemas, not models.
Exporting a model from a le is called the export model pattern. The export model pattern is
limited because you can only use one connection.
// The alternative to the export model pattern is the export schema pattern.
module.exports = userSchema;
// Because if you export a model as shown below, the model will be scoped
// to Mongoose's default connection.
// module.exports = mongoose.model('User', userSchema);
If you use the export schema pattern, you still need to create models somewhere. There are two
common patterns. First is to export a connection and register the models on the connection in
the le:
// connections/fast.js
const mongoose = require('mongoose');
module.exports = conn;
// connections/slow.js
const mongoose = require('mongoose');
module.exports = conn;
conn.model('User', require('../schemas/user'));
conn.model('PageView', require('../schemas/pageView'));
return conn;
};
Connection Pools
Next Up
Models
Models are fancy constructors compiled from Schema de nitions. An instance of a model is
called a document. Models are responsible for creating and reading documents from the
underlying MongoDB database.
The rst argument is the singular name of the collection your model is for. Mongoose
automatically looks for the plural, lowercased version of your model name. Thus, for the example
above, the model Tank is for the tanks collection in the database.
Note: The .model() function makes a copy of schema . Make sure that you've added everything
you want to schema , including hooks, before calling .model() !
Constructing Documents
An instance of a model is called a document. Creating them and saving to the database is easy.
// or
});
Note that no tanks will be created/removed until the connection your model uses is open. Every
model has an associated connection. When you use mongoose.model() , your model will use the
default mongoose connection.
mongoose.connect('mongodb://127.0.0.1/gettingstarted');
If you create a custom connection, use that connection's model() function instead.
Querying
Finding documents is easy with Mongoose, which supports the rich query syntax of MongoDB.
Documents can be retrieved using a model 's nd, ndById, ndOne, or where static functions.
See the chapter on queries for more details on how to use the Query api.
Deleting
Models have static deleteOne() and deleteMany() functions for removing all documents
matching the given filter .
Updating
Each model has its own update method for modifying documents in the database without
returning them to your application. See the API docs for more detail.
If you want to update a single document in the db and return it to your application, use
ndOneAndUpdate instead.
Change Streams
Change streams provide a way for you to listen to all inserts and updates going through your
MongoDB database. Note that change streams do not work unless you're connected to a
MongoDB replica set. This means that if you want to use change streams, your MongoDB deployment must be a replica
set, and not just a single instance. This is because change streams rely on the ability to detect
changes by comparing the data between replica set members, so it only works in replica set
configuration.
async function run() {
// Create a new mongoose model
const personSchema = new mongoose.Schema({
name: String
});
const Person = mongoose.model('Person', personSchema);
// Create a change stream. The 'change' event gets emitted when there's a
// change in the database
Person.watch().
on('change', data => console.log(new Date(), data));
The output from the above async function will look like what you see below.
You can read more about change streams in mongoose in this blog post.
Views
MongoDB Views are essentially read-only collections that contain data computed from other
collections using aggregations. In Mongoose, you should de ne a separate Model for each of
your Views. You can also create a View using createCollection() .
The following example shows how you can create a new RedactedUser View on a User Model
that hides potentially sensitive information, like name and email.
await User.create([
{ name: 'John Smith', email: '[email protected]', roles: ['user'] },
{ name: 'Bill James', email: '[email protected]', roles: ['user', 'admin'] }
]);
Note that Mongoose does not currently enforce that Views are read-only. If you attempt to
save() a document from a View, you will get an error from the MongoDB server.
Yet more
The API docs cover many additional methods available like count, mapReduce, aggregate, and
more.
Next Up
Now that we've covered Models , let's take a look at Documents.
mongoose
Documents
Documents vs Models
Retrieving
Updating Using save()
Updating Using Queries
Validating
Overwriting
Documents vs Models
Document and Model are distinct classes in Mongoose. The Model class is a subclass of the
Document class. When you use the Model constructor, you create a new document.
In Mongoose, a "document" generally means an instance of a model. You should not have to
create an instance of the Document class without going through a model.
Retrieving
When you load documents from MongoDB using model functions like findOne() , you get a
Mongoose document back.
doc.name = 'foo';
The save() method returns a promise. If save() succeeds, the promise resolves to the
document that was saved.
doc.save().then(savedDoc => {
savedDoc === doc; // true
});
If the document with the corresponding _id is not found, Mongoose will report a
DocumentNotFoundError :
doc.name = 'foo';
await doc.save(); // Throws DocumentNotFoundError
For cases when save() isn't exible enough, Mongoose lets you create your own MongoDB
updates with casting, middleware, and limited validation.
Validating
Documents are casted and validated before they are saved. Mongoose rst casts values to the
speci ed type and then validates them. Internally, Mongoose calls the document's validate()
method before saving.
const schema = new Schema({ name: String, age: { type: Number, min: 0 } });
const Person = mongoose.model('Person', schema);
Mongoose also supports limited validation on updates using the runValidators option.
Mongoose casts parameters to query functions like findOne() , updateOne() by default.
However, Mongoose does not run validation on query function parameters by default. You need
to set runValidators: true for Mongoose to validate.
means by default type is checked of the
data ,but validations are not checked like min,
maxLength,validate etc
// Cast to number failed for value "bar" at path "age"
await Person.updateOne({}, { age: 'bar' });
Overwriting
There are 2 di erent ways to overwrite a document (replacing all keys in the document). One way
is to use the Document#overwrite() function followed by save() .
Next Up
Subdocuments
Subdocuments are documents embedded in other documents. In Mongoose, this means you can
nest schemas in other schemas. Mongoose has two distinct notions of subdocuments: arrays of
subdocuments and single nested subdocuments.
Note that populated documents are not subdocuments in Mongoose. Subdocument data is
embedded in the top-level document. Referenced documents are separate top-level documents.
What is a Subdocument?
Subdocuments versus Nested Paths
Subdocument Defaults
Finding a Subdocument
Adding Subdocs to Arrays
Removing Subdocs
Parents of Subdocs
Alternate declaration syntax for arrays
What is a Subdocument?
Subdocuments are similar to normal documents. Nested schemas can have middleware, custom
validation logic, virtuals, and any other feature top-level schemas can use. The major di erence is
that subdocuments are not saved individually, they are saved whenever their top-level parent
document is saved.
Subdocuments have save and validate middleware just like top-level documents. Calling
save() on the parent document triggers the save() middleware for all its subdocuments, and
the same for validate() middleware.
childSchema.pre('validate', function(next) {
console.log('2');
next();
});
childSchema.pre('save', function(next) {
console.log('3');
next();
});
parentSchema.pre('validate', function(next) {
console.log('1');
next();
});
parentSchema.pre('save', function(next) {
console.log('4');
next();
});
In Mongoose, nested paths are subtly di erent from subdocuments. For example, below are two
schemas: one with child as a subdocument, and one with child as a nested path.
// Subdocument
const subdocumentSchema = new mongoose.Schema({
child: new mongoose.Schema({ name: String, age: Number })
});
const Subdoc = mongoose.model('Subdoc', subdocumentSchema);
// Nested path
const nestedSchema = new mongoose.Schema({
child: { name: String, age: Number }
});
const Nested = mongoose.model('Nested', nestedSchema);
These two schemas look similar, and the documents in MongoDB will have the same structure
with both schemas. But there are a few Mongoose-speci c di erences:
First, instances of Nested never have child === undefined . You can always set subproperties
of child , even if you don't set the child property. But instances of Subdoc can have child
=== undefined .
Subdocument Defaults
Subdocument paths are unde ned by default, and Mongoose does not apply subdocument
defaults unless you set the subdocument path to a non-nullish value.
However, if you set doc.child to any object, Mongoose will apply the age default if necessary.
doc.child = {};
// Mongoose applies the `age` default:
doc.child.age; // 0
Mongoose applies defaults recursively, which means there's a nice workaround if you want to
make sure Mongoose applies subdocument defaults: make the subdocument path default to an
empty object.
Finding a Subdocument
Each subdocument has an _id by default. Mongoose document arrays have a special id method
for searching a document array to nd a document with a given _id .
MongooseArray methods such as push, unshift, addToSet, and others cast arguments to their
proper types transparently:
// create a comment
parent.children.push({ name: 'Liesl' });
const subdoc = parent.children[0];
console.log(subdoc) // { _id: '501d86090d371bab2c0341c5', name: 'Liesl' }
subdoc.isNew; // true
parent.save(function (err) {
if (err) return handleError(err)
console.log('Success!');
});
You can also create a subdocument without adding it to an array by using the create() method
of Document Arrays.
Removing Subdocs
Each subdocument has its own remove method. For an array subdocument, this is equivalent to
calling .pull() on the subdocument. For a single nested subdocument, remove() is equivalent
to setting the subdocument to null .
// Equivalent to `parent.children.pull(_id)`
parent.children.id(_id).remove();
// Equivalent to `parent.child = null`
parent.child.remove();
parent.save(function (err) {
if (err) return handleError(err);
console.log('the subdocs were removed');
});
Parents of Subdocs
Sometimes, you need to get the parent of a subdoc. You can access the parent using the
parent() function.
If you have a deeply nested subdoc, you can access the top-level document using the
ownerDocument() function.
If you create a schema with an array of objects, Mongoose will automatically convert the object to
a schema for you:
Next Up
Queries
Mongoose models provide several static helper functions for CRUD operations. Each of these
functions returns a mongoose Query object.
Model.deleteMany()
Model.deleteOne()
Model.find()
Model.findById()
Model.findByIdAndDelete()
Model.findByIdAndRemove()
Model.findByIdAndUpdate()
Model.findOne()
Model.findOneAndDelete()
Model.findOneAndRemove()
Model.findOneAndReplace()
Model.findOneAndUpdate()
Model.replaceOne()
Model.updateMany()
Model.updateOne()
A mongoose query can be executed in one of two ways. First, if you pass in a callback function,
Mongoose will execute the query asynchronously and pass the results to the callback .
A query also has a .then() function, and thus can be used as a promise.
Executing
Queries are Not Promises
References to other documents
Streaming
Versus Aggregation
Executing
When executing a query with a callback function, you specify your query as a JSON document.
The JSON document's syntax is the same as the MongoDB shell.
// find each person with a last name matching 'Ghost', selecting the `name` and `occupation
Person.findOne({ 'name.last': 'Ghost' }, 'name occupation', function (err, person) {
if (err) return handleError(err);
// Prints "Space Ghost is a talk show host".
console.log('%s %s is a %s.', person.name.first, person.name.last,
person.occupation);
});
Mongoose executed the query and passed the results to callback . All callbacks in Mongoose
use the pattern: callback(error, result) . If an error occurs executing the query, the error
parameter will contain an error document, and result will be null. If the query is successful, the
error parameter will be null, and the result will be populated with the results of the query.
Anywhere a callback is passed to a query in Mongoose, the callback follows the pattern
callback(error, results) . What results is depends on the operation: For findOne() it is a
potentially-null single document, find() a list of documents, count() the number of
documents, update() the number of documents a ected, etc. The API docs for Models provide
more detail on what is passed to the callbacks.
In the above code, the query variable is of type Query. A Query enables you to build up a query
using chaining syntax, rather than specifying a JSON object. The below 2 examples are equivalent.
A full list of Query helper functions can be found in the API docs.
Mongoose queries are not promises. They have a .then() function for co and async/await as a
convenience. However, unlike promises, calling a query's .then() can execute the query
multiple times.
For example, the below code will execute 3 updateMany() calls, one because of the callback, and
two because .then() is called twice.
Don't mix using callbacks and promises with queries, or you may end up with duplicate
operations. That's because passing a callback to a query function immediately executes the
query, and calling then() executes the query again.
Mixing promises and callbacks can lead to duplicate entries in arrays. For example, the below
code inserts 2 entries into the tags array, not just 1.
// Because there's both `await` **and** a callback, this `updateOne()` executes twice
// and thus pushes the same string into `tags` twice.
const update = { $push: { tags: ['javascript'] } };
await BlogPost.updateOne({ title: 'Introduction to Promises' }, update, (err, res) => {
console.log(res);
});
There are no joins in MongoDB but sometimes we still want references to documents in other
collections. This is where population comes in. Read more about how to include documents from
other collections in your query results here.
Streaming
You can stream query results from MongoDB. You need to call the Query#cursor() function to
return an instance of QueryCursor.
for (let doc = await cursor.next(); doc != null; doc = await cursor.next()) {
console.log(doc); // Prints documents one at a time
}
Iterating through a Mongoose query using async iterators also creates a cursor.
Cursors are subject to cursor timeouts. By default, MongoDB will close your cursor after 10
minutes and subsequent next() calls will result in a MongoServerError: cursor id 123 not
found error. To override this, set the noCursorTimeout option on your cursor.
However, cursors can still time out because of session idle timeouts. So even a cursor with
noCursorTimeout set will still time out after 30 minutes of inactivity. You can read more about
working around session idle timeouts in the MongoDB documentation.
Versus Aggregation
Aggregation can do many of the same things that queries can. For example, below is how you can
use aggregate() to nd docs where name.last = 'Ghost' :
However, just because you can use aggregate() doesn't mean you should. In general, you
should use queries where possible, and only use aggregate() when you absolutely need to.
Unlike query results, Mongoose does not hydrate() aggregation results. Aggregation results are
always POJOs, not Mongoose documents.
Also, unlike query lters, Mongoose also doesn't cast aggregation pipelines. That means you're
responsible for ensuring the values you pass in to an aggregation pipeline have the correct type.
// Does **not** find the `Person`, because Mongoose doesn't cast aggregation
// pipelines.
const aggRes = await Person.aggregate([{ $match: { _id: idString } }])
Sorting
Sorting is how you can ensure you query results come back in the desired order.
When sorting with mutiple elds, the order of the sort keys determines what key MongoDB
server sorts by rst.
await Person.find().sort({ age: 1, weight: -1 }); // returns age starting from 0, but while
You can view the output of a single run of this block below. As you can see, age is sorted from 0
to 2 but when age is equal, sorts by weight.
[
{
_id: new ObjectId("63a335a6b9b6a7bfc186cb37"),
age: 0,
name: 'Test2',
weight: 67,
__v: 0
},
{
_id: new ObjectId("63a335a6b9b6a7bfc186cb35"),
age: 1,
name: 'Test1',
weight: 99,
__v: 0
},
{
_id: new ObjectId("63a335a6b9b6a7bfc186cb39"),
age: 1,
name: 'Test3',
weight: 73,
__v: 0
},
{
_id: new ObjectId("63a335a6b9b6a7bfc186cb33"),
age: 2,
name: 'Test0',
weight: 65,
__v: 0
},
{
_id: new ObjectId("63a335a6b9b6a7bfc186cb3b"),
age: 2,
name: 'Test4',
weight: 62,
__v: 0
}
]
Next Up
Validation
Before we get into the speci cs of validation syntax, please keep the following rules in mind:
let error;
try {
await cat.save();
} catch (err) {
error = err;
}
assert.equal(error.errors['name'].message,
'Path `name` is required.');
error = cat.validateSync();
assert.equal(error.errors['name'].message,
'Path `name` is required.');
Built-in Validators
Custom Error Messages
The unique Option is Not a Validator
Custom Validators
Async Custom Validators
Validation Errors
Cast Errors
Global SchemaType Validation
Required Validators On Nested Objects
Update Validators
Update Validators and this
Update Validators Only Run On Updated Paths
Update Validators Only Run For Some Operations
Built-in Validators
All SchemaTypes have the built-in required validator. The required validator uses the
SchemaType's checkRequired() function to determine if the value satis es the required
validator.
Numbers have min and max validators.
Strings have enum , match , minLength , and maxLength validators.
Each of the validator links above provide more information about how to enable them and
customize their error messages.
badBreakfast.bacon = 5;
badBreakfast.drink = null;
error = badBreakfast.validateSync();
assert.equal(error.errors['drink'].message, 'Path `drink` is required.');
badBreakfast.bacon = null;
error = badBreakfast.validateSync();
assert.equal(error.errors['bacon'].message, 'Why no bacon?');
You can con gure the error message for individual validators in your schema. There are two
equivalent ways to set the validator error message:
Mongoose also supports rudimentary templating for error messages. Mongoose replaces
{VALUE} with the value being validated.
A common gotcha for beginners is that the unique option for schemas is not a validator. It's a
convenient helper for building MongoDB unique indexes. See the FAQ for more information.
Custom Validators
If the built-in validators aren't enough, you can de ne custom validators to suit your needs.
Custom validation is declared by passing a validation function. You can nd detailed instructions
on how to do this in the SchemaType#validate() API docs.
user.phone = '555.0123';
error = user.validateSync();
assert.equal(error.errors['phone'].message,
'555.0123 is not a valid phone number!');
user.phone = '';
error = user.validateSync();
assert.equal(error.errors['phone'].message,
'User phone number required');
user.phone = '201-555-0123';
// Validation succeeds! Phone number is defined
// and fits `DDD-DDD-DDDD`
error = user.validateSync();
assert.equal(error, null);
Custom validators can also be asynchronous. If your validator function returns a promise (like an
async function), mongoose will wait for that promise to settle. If the returned promise rejects, or
ful lls with the value false , Mongoose will consider that a validation error.
user.email = '[email protected]';
user.name = 'test';
let error;
try {
await user.validate();
if a promise gets rejected promise does not return the error,
} catch (err) { but error is thrown which is catched in the catch() block or .catch()
error = err; in try block we can use await
}
assert.ok(error);
assert.equal(error.errors['name'].message, 'Oops!');
assert.equal(error.errors['email'].message, 'Email validation failed');
Validation Errors
Errors returned after failed validation contain an errors object whose values are
ValidatorError objects. Each ValidatorError has kind , path , value , and message properties.
A ValidatorError also may have a reason property. If an error was thrown in the validator, this
property will contain the error that was thrown.
const toy = new Toy({ color: 'Green', name: 'Power Ranger' });
let error;
try {
await toy.save();
} catch(err) {
error = err;
}
assert.equal(error.name, 'ValidationError');
Cast Errors
Before running validators, Mongoose attempts to coerce values to the correct type. This process
is called casting the document. If casting fails for a given path, the error.errors object will
contain a CastError object.
Casting runs before validation, and validation does not run if casting fails. That means your
custom validators may assume v is null , undefined , or an instance of the type speci ed in
your schema.
err.errors['numWheels'].name; // 'CastError'
// 'Cast to Number failed for value "not a number" at path "numWheels"'
err.errors['numWheels'].message;
In addition to de ning custom validators on individual schema paths, you can also con gure a
custom validator to run on every instance of a given SchemaType . For example, the following
code demonstrates how to make empty string '' an invalid value for all string paths.
De ning validators on nested objects in mongoose is tricky, because nested objects are not fully
edged paths.
assert.throws(function() {
// This throws an error, because 'name' isn't a full fledged path
personSchema.path('name').required(true);
}, /Cannot.*'required'/);
Update Validators
In the above examples, you learned about document validation. Mongoose also supports
validation for update() , updateOne() , updateMany() , and findOneAndUpdate() operations.
Update validators are o by default - you need to specify the runValidators option.
To turn on update validators, set the runValidators option for update() , updateOne() ,
updateMany() , or findOneAndUpdate() . Be careful: update validators are o by default because
they have several caveats.
Toy.schema.path('color').validate(function(value) {
return /red|green|blue/i.test(value);
}, 'Invalid color');
let error;
try {
await Toy.updateOne({}, { color: 'not a color' }, opts);
} catch (err) {
error = err;
}
There are a couple of key di erences between update validators and document validators. In the
color validation function below, this refers to the document being validated when using
document validation. However, when running update validators, this refers to the query object
instead of the document. Because queries have a neat .get() function, you can get the updated
value of the property you want.
toySchema.path('color').validate(function(value) {
// When running in `validate()` or `validateSync()`, the
// validator can access the document using `this`.
// When running with update validators, `this` is the Query,
// **not** the document being updated!
// Queries have a `get()` method that lets you get the
// updated value.
if (this.get('name') && this.get('name').toLowerCase().indexOf('red') !== -1) {
return value === 'red';
}
return true;
});
const toy = new Toy({ color: 'green', name: 'Red Power Ranger' });
// Validation failed: color: Validator failed for path `color` with value `green`
let error = toy.validateSync();
assert.ok(error.errors['color']);
error = null;
try {
await Toy.updateOne({}, update, opts);
} catch (err) {
error = err;
}
// Validation failed: color: Validator failed for path `color` with value `green`
assert.ok(error);
The other key di erence is that update validators only run on the paths speci ed in the update.
For instance, in the below example, because 'name' is not speci ed in the update operation,
update validation will succeed.
When using update validators, required validators only fail when you try to explicitly $unset
the key.
One nal detail worth noting: update validators only run on the following update operators:
$set
$unset
$push
$addToSet
$pull
$pullAll
For instance, the below update will succeed, regardless of the value of number , because update
validators ignore $inc .
Also, $push , $addToSet , $pull , and $pullAll validation does not run any validation on the
array itself, only individual elements of the array.
// Update validators won't check this, so you can still `$push` 2 elements
// onto the array, so long as they don't have a `message` that's too long.
testSchema.path('arr').validate(function(v) {
return v.length < 2;
});
// This will never error either even though the array will have at
// least 2 elements.
update = { $push: [{ message: 'hello' }, { message: 'world' }] };
await Test.updateOne({}, update, opts);
Next Up
Middleware
Middleware (also called pre and post hooks) are functions which are passed control during
execution of asynchronous functions. Middleware is speci ed on the schema level and is useful
for writing plugins.
Types of Middleware
Pre
Errors in Pre Hooks
Post
Asynchronous Post Hooks
De ne Middleware Before Compiling Models
Save/Validate Hooks
Naming Con icts
Notes on ndAndUpdate() and Query Middleware
Error Handling Middleware
Aggregation Hooks
Synchronous Hooks
Types of Middleware
validate
save
remove
updateOne
deleteOne
init (note: init hooks are synchronous)
Query middleware is supported for the following Query functions. Query middleware executes
when you call exec() or then() on a Query object, or await on a Query object. In query
middleware functions, this refers to the query.
count
countDocuments
deleteMany
deleteOne
estimatedDocumentCount
nd
ndOne
ndOneAndDelete
ndOneAndRemove
ndOneAndReplace
ndOneAndUpdate
remove
replaceOne
update
updateOne
updateMany
aggregate
Model middleware is supported for the following model functions. Don't confuse model
middleware and document middleware: model middleware hooks into static functions on a
Model class, document middleware hooks into methods on a Model class. In model middleware
functions, this refers to the model.
insertMany
All middleware types support pre and post hooks. How pre and post hooks work is described in
more detail below.
Note: If you specify schema.pre('remove') , Mongoose will register this middleware for
doc.remove() by default. If you want your middleware to run on Query.remove() use
schema.pre('remove', { query: true, document: false }, fn) .
mainSchema.pre('findOneAndUpdate', function () {
console.log('Middleware on parent document'); // Will be executed
});
childSchema.pre('findOneAndUpdate', function () {
console.log('Middleware on subdocument'); // Will not be executed
});
Pre
Pre middleware functions are executed one after another, when each middleware calls next .
In mongoose 5.x, instead of calling next() manually, you can use a function that returns a
promise. In particular, you can use async/await .
schema.pre('save', function() {
return doStuff().
then(() => doMoreStuff());
});
If you use next() , the next() call does not stop the rest of the code in your middleware
function from executing. Use the early return pattern to prevent the rest of your middleware
function from running when you call next() .
Use Cases
Middleware are useful for atomizing model logic. Here are some other ideas:
complex validation
removing dependent documents (removing a user removes all their blogposts)
asynchronous defaults
asynchronous tasks that a certain action triggers
If any pre hook errors out, mongoose will not execute subsequent middleware or the hooked
function. Mongoose will instead pass an error to the callback and/or reject the returned promise.
There are several ways to report an error in middleware:
schema.pre('save', function(next) {
const err = new Error('something went wrong');
// If you call `next()` with an argument, that argument is assumed to be
// an error.
next(err);
});
schema.pre('save', function() {
// You can also return a promise that rejects
return new Promise((resolve, reject) => {
reject(new Error('something went wrong'));
});
});
schema.pre('save', function() {
// You can also throw a synchronous error
throw new Error('something went wrong');
});
// later...
// Changes will not be persisted to MongoDB because a pre hook errored out
myDoc.save(function(err) {
console.log(err.message); // something went wrong
});
Calling next() multiple times is a no-op. If you call next() with an error err1 and then throw
an error err2 , mongoose will report err1 .
Post middleware
post middleware are executed after the hooked method and all of its pre middleware have
completed.
schema.post('init', function(doc) {
console.log('%s has been initialized from the db', doc._id);
});
schema.post('validate', function(doc) {
console.log('%s has been validated (but not saved yet)', doc._id);
});
schema.post('save', function(doc) {
console.log('%s has been saved', doc._id);
});
schema.post('remove', function(doc) {
console.log('%s has been removed', doc._id);
});
If your post hook function takes at least 2 parameters, mongoose will assume the second
parameter is a next() function that you will call to trigger the next middleware in the sequence.
Calling pre() or post() after compiling a model does not work in Mongoose in general. For
example, the below pre('save') middleware will not re.
This means that you must add all middleware and plugins before calling mongoose.model() . The
below script will print out "Hello from pre save":
As a consequence, be careful about exporting Mongoose models from the same le that you
de ne your schema. If you choose to use this pattern, you must de ne global plugins before
calling require() on your model le.
// Once you `require()` this file, you can no longer add any middleware
// to this schema.
module.exports = mongoose.model('User', schema);
Save/Validate Hooks
The save() function triggers validate() hooks, because mongoose has a built-in
pre('save') hook that calls validate() . This means that all pre('validate') and
post('validate') hooks get called before any pre('save') hooks.
schema.pre('validate', function() {
console.log('this gets printed first');
});
schema.post('validate', function() {
console.log('this gets printed second');
});
schema.pre('save', function() {
console.log('this gets printed third');
});
schema.post('save', function() {
console.log('this gets printed fourth');
});
// Prints "Removing!"
doc.remove();
You can pass options to Schema.pre() and Schema.post() to switch whether Mongoose calls
your remove() hook for Document.remove() or Model.remove() . Note here that you need to
set both document and query properties in the passed object:
// Only query middleware. This will get called when you do `Model.remove()`
// but not `doc.remove()`.
schema.pre('remove', { query: true, document: false }, function() {
console.log('Removing!');
});
Pre and post save() hooks are not executed on update() , findOneAndUpdate() , etc. You can
see a more detailed discussion why in this GitHub issue. Mongoose 4.0 introduced distinct hooks
for these functions.
schema.pre('find', function() {
console.log(this instanceof mongoose.Query); // true
this.start = Date.now();
});
schema.post('find', function(result) {
console.log(this instanceof mongoose.Query); // true
// prints returned documents
console.log('find() returned ' + JSON.stringify(result));
// prints number of milliseconds the query took
console.log('find() took ' + (Date.now() - this.start) + ' milliseconds');
});
Query middleware di ers from document middleware in a subtle but important way: in
document middleware, this refers to the document being updated. In query middleware,
mongoose doesn't necessarily have a reference to the document being updated, so this refers
to the query object rather than the document being updated.
For instance, if you wanted to add an updatedAt timestamp to every updateOne() call, you
would use the following pre hook.
schema.pre('updateOne', function() {
this.set({ updatedAt: new Date() });
});
New in 4.5.0
Middleware execution normally stops the rst time a piece of middleware calls next() with an
error. However, there is a special kind of post middleware called "error handling middleware"
that executes speci cally when an error occurs. Error handling middleware is useful for reporting
errors and making error messages more readable.
Error handling middleware is de ned as middleware that takes one extra parameter: the 'error'
that occurred as the rst parameter to the function. Error handling middleware can then
transform the error however you want.
// Handler **must** take 3 parameters: the error that occurred, the document
// in question, and the `next()` function
schema.post('save', function(error, doc, next) {
if (error.name === 'MongoServerError' && error.code === 11000) {
next(new Error('There was a duplicate key error'));
} else {
next();
}
});
Error handling middleware also works with query middleware. You can also de ne a post
update() hook that will catch MongoDB duplicate key errors.
// The same E11000 error can occur when you call `update()`
// This function **must** take 3 parameters. If you use the
// `passRawResult` function, this function **must** take 4
// parameters
schema.post('update', function(error, res, next) {
if (error.name === 'MongoServerError' && error.code === 11000) {
next(new Error('There was a duplicate key error'));
} else {
next(); // The `update()` call will still error out.
}
});
Error handling middleware can transform an error, but it can't remove the error. Even if you call
next() with no error as shown above, the function call will still error out.
Aggregation Hooks
You can also de ne hooks for the Model.aggregate() function. In aggregation middleware
functions, this refers to the Mongoose Aggregate object. For example, suppose you're
implementing soft deletes on a Customer model by adding an isDeleted property. To make
sure aggregate() calls only look at customers that aren't soft deleted, you can use the below
middleware to add a $match stage to the beginning of each aggregation pipeline.
customerSchema.pre('aggregate', function() {
// Add a $match state to the beginning of each pipeline.
this.pipeline().unshift({ $match: { isDeleted: { $ne: true } } });
});
The Aggregate#pipeline() function lets you access the MongoDB aggregation pipeline that
Mongoose will send to the MongoDB server. It is useful for adding stages to the beginning of the
pipeline from middleware.
Synchronous Hooks
Certain Mongoose hooks are synchronous, which means they do not support functions that
return promises or receive a next() callback. Currently, only init hooks are synchronous,
because the init() function is synchronous. Below is an example of using pre and post init
hooks.
To report an error in an init hook, you must throw a synchronous error. Unlike all other
middleware, init middleware does not handle promise rejections.
Next Up
Now that we've covered middleware, let's take a look at Mongoose's approach to faking JOINs
with its query population helper.
mongoose
Populate
MongoDB has the join-like $lookup aggregation operator in versions >= 3.2. Mongoose has a
more powerful alternative called populate() , which lets you reference documents in other
collections.
Population is the process of automatically replacing the speci ed paths in the document with
document(s) from other collection(s). We may populate a single document, multiple documents,
a plain object, multiple plain objects, or all objects returned from a query. Let's look at some
examples.
So far we've created two Models. Our Person model has its stories eld set to an array of
ObjectId s. The ref option is what tells Mongoose which model to use during population, in
our case the Story model. All _id s we store here must be document _id s from the Story
model.
Note: ObjectId , Number , String , and Buffer are valid for use as refs. However, you should
use ObjectId unless you are an advanced user and have a good reason for doing so.
Saving Refs
Population
Checking Whether a Field is Populated
Setting Populated Fields
What If There's No Foreign Document?
Field Selection
Populating Multiple Paths
Query conditions and other options
Refs to children
Populating an existing document
Populating multiple existing documents
Populating across multiple levels
Populating across Databases
Dynamic References via `refPath`
Populate Virtuals
Populate Virtuals: The Count Option
Populate Virtuals: The Match Option
Populating Maps
Populate in Middleware
Populating Multiple Paths in Middleware
Transform populated documents
Saving refs
Saving refs to other documents works the same way you normally save properties, just assign the
_id value:
author.save(function (err) {
if (err) return handleError(err);
story1.save(function (err) {
if (err) return handleError(err);
// that's it!
});
});
Population
So far we haven't done anything much di erent. We've merely created a Person and a Story .
Now let's take a look at populating our story's author using the query builder:
Story.
findOne({ title: 'Casino Royale' }).
populate('author').
exec(function (err, story) {
if (err) return handleError(err);
console.log('The author is %s', story.author.name);
// prints "The author is Ian Fleming"
});
Populated paths are no longer set to their original _id , their value is replaced with the
mongoose document returned from the database by performing a separate query before
returning the results.
Arrays of refs work the same way. Just call the populate method on the query and an array of
documents will be returned in place of the original _id s.
You can manually populate a property by setting it to a document. The document must be an
instance of the model your ref property refers to.
You can call the populated() function to check whether a eld is populated. If populated()
returns a truthy value, you can assume the eld is populated.
story.populated('author'); // truthy
A common reason for checking whether a path is populated is getting the author id. However,
for your convenience, Mongoose adds a _id getter to ObjectId instances so you can use
story.author._id regardless of whether author is populated.
story.populated('author'); // truthy
story.author._id; // ObjectId
Mongoose populate doesn't behave like conventional SQL joins. When there's no document,
story.author will be null . This is analogous to a left join in SQL.
If you have an array of authors in your storySchema , populate() will give you an empty array
instead.
// Later
Field Selection
What if we only want a few speci c elds returned for the populated documents? This can be
accomplished by passing the usual eld name syntax as the second argument to the populate
method:
Story.
findOne({ title: /casino royale/i }).
populate('author', 'name'). // only return the Persons name
exec(function (err, story) {
if (err) return handleError(err);
Story.
find(...).
populate('fans').
populate('author').
exec();
If you call populate() multiple times with the same path, only the last one will take e ect.
// The 2nd `populate()` call below overwrites the first because they
// both populate 'fans'.
Story.
find().
populate({ path: 'fans', select: 'name' }).
populate({ path: 'fans', select: 'email' });
// The above is equivalent to:
Story.find().populate({ path: 'fans', select: 'email' });
What if we wanted to populate our fans array based on their age and select just their names?
Story.
find().
populate({
path: 'fans',
match: { age: { $gte: 21 } },
// Explicitly exclude `_id`, see http://bit.ly/2aEfTdB
select: 'name -_id'
}).
exec();
The match option doesn't lter out Story documents. If there are no documents that satisfy
match , you'll get a Story document with an empty fans array.
For example, suppose you populate() a story's author and the author doesn't satisfy match .
Then the story's author will be null .
In general, there is no way to make populate() lter stories based on properties of the story's
author . For example, the below query won't return any results, even though author is
populated.
If you want to lter stories by their author's name, you should use denormalization.
Populate does support a limit option, however, it currently does not limit on a per-document
basis for backwards compatibility. For example, suppose you have 2 stories:
Story.create([
{ title: 'Casino Royale', fans: [1, 2, 3, 4, 5, 6, 7, 8] },
{ title: 'Live and Let Die', fans: [9, 10] }
]);
If you were to populate() using the limit option, you would nd that the 2nd story has 0 fans:
That's because, in order to avoid executing a separate query for each document, Mongoose
instead queries for fans using numDocuments * limit as the limit. If you need the correct
limit , you should use the perDocumentLimit option (new in Mongoose 5.9.0). Just keep in
mind that populate() will execute a separate query for each story, which may cause
populate() to be slower.
Refs to children
We may nd however, if we use the author object, we are unable to get a list of the stories. This
is because no story objects were ever 'pushed' onto author.stories .
There are two perspectives here. First, you may want the author to know which stories are
theirs. Usually, your schema should resolve one-to-many relationships by having a parent pointer
in the 'many' side. But, if you have a good reason to want an array of child pointers, you can
push() documents onto the array as shown below.
story1.save()
author.stories.push(story1);
author.save(callback);
Person.
findOne({ name: 'Ian Fleming' }).
populate('stories'). // only works if we pushed refs to children
exec(function (err, person) {
if (err) return handleError(err);
console.log(person);
});
It is debatable that we really want two sets of pointers as they may get out of sync. Instead we
could skip populating and directly find() the stories we are interested in.
Story.
find({ author: author._id }).
exec(function (err, stories) {
if (err) return handleError(err);
console.log('The stories are an array: ', stories);
});
The documents returned from query population become fully functional, remove able, save able
documents unless the lean option is speci ed. Do not confuse them with sub docs. Take caution
when calling its remove method because you'll be removing it from the database, not just the
array.
If you have an existing mongoose document and want to populate some of its paths, you can use
the Document#populate() method.
person.populated('stories'); // null
The Document#populate() method does not support chaining. You need to call populate()
multiple times, or with an array of paths, to populate multiple paths
If we have one or many mongoose documents or even plain objects (like mapReduce output), we
may populate them using the Model.populate() method. This is what Document#populate() and
Query#populate() use to populate documents.
Say you have a user schema which keeps track of the user's friends.
Populate lets you get a list of a user's friends, but what if you also wanted a user's friends of
friends? Specify the populate option to tell mongoose to populate the friends array of all the
user's friends:
User.
findOne({ name: 'Val' }).
populate({
path: 'friends',
// Get friends of friends - populate the 'friends' array for every friend
populate: { path: 'friends' }
});
Let's say you have a schema representing events, and a schema representing conversations.
Each event has a corresponding conversation thread.
In the above example, events and conversations are stored in separate MongoDB databases.
String ref will not work in this situation, because Mongoose assumes a string ref refers to a
model name on the same connection. In the above example, the conversation model is
registered on db2 , not db1 .
// Works
const events = await Event.
find().
populate('conversation');
If you don't have access to the model instance when de ning your eventSchema , you can also
pass the model instance as an option to populate() .
Mongoose can also populate from multiple collections based on the value of a property in the
document. Let's say you're building a schema for storing comments. A user may comment on
either a blog post or a product.
The refPath option is a more sophisticated alternative to ref . If ref is a string, Mongoose will
always query the same model to nd the populated subdocs. With refPath , you can con gure
what model Mongoose uses for each document.
const book = await Product.create({ name: 'The Count of Monte Cristo' });
const post = await BlogPost.create({ title: 'Top 10 French Novels' });
// The below `populate()` works even though one comment references the
// 'Product' collection and the other references the 'BlogPost' collection.
const comments = await Comment.find().populate('doc').sort({ body: 1 });
comments[0].doc.name; // "The Count of Monte Cristo"
comments[1].doc.title; // "Top 10 French Novels"
// ...
De ning separate blogPost and product properties works for this simple example. But, if you
decide to allow users to also comment on articles or other comments, you'll need to add more
properties to your schema. You'll also need an extra populate() call for every property, unless
you use mongoose-autopopulate. Using refPath means you only need 2 schema paths and one
populate() call regardless of how many models your commentSchema can point to.
Populate Virtuals
So far you've only populated based on the _id eld. However, that's sometimes not the right
choice. For example, suppose you have 2 models: Author and BlogPost .
The above is an example of bad schema design. Why? Suppose you have an extremely proli c
author that writes over 10k blog posts. That author document will be huge, over 12kb, and large
documents lead to performance issues on both server and client. The Principle of Least
Cardinality states that one-to-many relationships, like author to blog post, should be stored on
the "many" side. In other words, blog posts should store their author , authors should not store
all their posts .
Unfortunately, these two schemas, as written, don't support populating an author's list of blog
posts. That's where virtual populate comes in. Virtual populate means calling populate() on a
virtual property that has a ref option as shown below.
Keep in mind that virtuals are not included in toJSON() and toObject() output by default. If
you want populate virtuals to show up when using functions like Express' res.json() function
or console.log() , set the virtuals: true option on your schema's toJSON and toObject()
options.
If you're using populate projections, make sure foreignField is included in the projection.
Populate virtuals also support counting the number of documents with matching foreignField
as opposed to the documents themselves. Set the count option on your virtual:
// Later
const doc = await Band.findOne({ name: 'Motley Crue' }).
populate('numMembers');
doc.numMembers; // 2
Another option for Populate virtuals is match . This option adds an extra lter condition to the
query Mongoose uses to populate() :
// After population
const author = await Author.findOne().populate('posts');
You can also set the match option to a function. That allows con guring the match based on the
document being populated. For example, suppose you only want to populate blog posts whose
tags contain one of the author's favoriteTags .
AuthorSchema.virtual('posts', {
ref: 'BlogPost',
localField: '_id',
foreignField: 'author',
// Add an additional filter `{ tags: author.favoriteTags }` to the populate query
// Mongoose calls the `match` function with the document being populated as the
// first argument.
match: author => ({ tags: author.favoriteTags })
});
Populating Maps
Maps are a type that represents an object with arbitrary string keys. For example, in the below
schema, members is a map from strings to ObjectIds.
This map has a ref , which means you can use populate() to populate all the ObjectIds in the
map. Suppose you have the below band document:
You can populate() every element in the map by populating the special path members.$* . $*
is a special syntax that tells Mongoose to look at every key in the map.
You can also populate paths in maps of subdocuments using $* . For example, suppose you
have the below librarySchema :
Populate in Middleware
You can populate in either pre or post hooks. If you want to always populate a certain eld, check
out the mongoose-autopopulate plugin.
// Always `populate()` after `find()` calls. Useful if you want to selectively populate
// based on the docs found.
MySchema.post('find', async function(docs) {
for (let doc of docs) {
if (doc.isPublic) {
await doc.populate('user');
}
}
});
// `populate()` after saving. Useful for sending populated data back to the client in an
// update API endpoint
MySchema.post('save', function(doc, next) {
doc.populate('user').then(function() {
next();
});
});
Populating multiple paths in middleware can be helpful when you always want to populate some
elds. But, the implementation is just a tiny bit trickier than what you may think. Here's how you
may expect it to work:
However, this will not work. By default, passing multiple paths to populate() in the middleware
will trigger an in nite recursion, which means that it will basically trigger the same middleware
for all of the paths provided to the populate() method - For example,
this.populate('followers following') will trigger the same middleware for both followers
and following elds and the request will just be left hanging in an in nite loop.
To avoid this, we have to add the _recursed option, so that our middleware will avoid
populating recursively. The example below will make it work as expected.
You can manipulate populated documents using the transform option. If you specify a
transform function, Mongoose will call this function on every populated document in the result
wiwith two arguments: the populated document, and the original id used to populate the
document. This gives you more control over the result of the populate() execution. It is
especially useful when you're populating multiple documents.
The original motivation for the transform option was to give the ability to leave the unpopulated
_id if no document was found, instead of setting the value to null :
// With `transform`
doc = await Parent.findById(doc).populate([
{
path: 'child',
// If `doc` is null, use the original id instead
transform: (doc, id) => doc == null ? id : doc
}
]);
doc.child; // 634d1a5744efe65ae09142f9
doc.children; // [ 634d1a67ac15090a0ca6c0ea, { _id: 634d1a4ddb804d17d95d1c7f, name: 'Luke',
You can return any value from transform() . For example, you can use transform() to " atten"
populated documents as follows.
let doc = await Parent.create({ children: [ { name: 'Luke' }, { name: 'Leia' } ] });
Another use case for transform() is setting $locals values on populated documents to pass
parameters to getters and virtuals. For example, suppose you want to set a language code on
your document for internationalization purposes as follows.
You can set the language code on all populated exercises as follows: