A MongoDB White Paper

RDBMS to MongoDB Migration Guide

Considerations and Best Practices
June 2017
Table of Contents
Introduction 1

Organizing for Success 1

Schema Design 2
From Rigid Tables to Flexible and Dynamic BSON
Documents 3
Other Advantages of the Document Model 4
Joining Collections for Data Analytics 4
Defining the Document Schema 5
Modeling Relationships with Embedding and
Referencing 5
Embedding 6
Rerencing 6
Different Design Goals 6
Indexing 6
Index Types 7
Optimizing Performance With Indexes 8
Schema Evolution and the Impact on Schema Design 9

Application Integration 10
MongoDB Drivers and the API 10
Mapping SQL to MongoDB Syntax 10
MongoDB Aggregation Framework 10
MongoDB Connector for BI 11
Atomicity in MongoDB 11
Maintaining Strong Consistency 12
Write Durability 12
Implementing Validation & Constraints 13
Foreign Keys 13
Document Validation 13
Enforcing Constraints With Indexes 13

Migrating Data to MongoDB 14

Operational Agility at Scale 15

MongoDB Atlas: Database as a Service For MongoDB 15

MongoDB Stitch: Backend as a Service 16

Supporting Your Migration: MongoDB Services 16

MongoDB University 16
Community Resources and Consulting 16

Conclusion 16

We Can Help 16

Resources 17
Introduction

The relational database has been the foundation of

Organizing for Success
enterprise data management for over thirty years.

But the way we build and run applications today, coupled

Before considering technologies and architecture, a key to
with unrelenting growth in new data sources and growing
success is involving all key stakeholders for the application,
user loads are pushing relational databases beyond their
including the line of business, developers, data architects,
limits. This can inhibit business agility, limit scalability and
DBAs, and systems administrators. In some organizations,
strain budgets, compelling more and more organizations to
these roles may be combined.
migrate to alternatives like MongoDB or NoSQL
databases. The project team should work together to define business
and technical objectives, timelines, and responsibilities,
As illustrated in Figure 1, enterprises from a variety of
meeting regularly to monitor progress and address any
industries have migrated successfully from relational
issues.
database management systems (RDBMS) to MongoDB for
myriad applications.
There are a range of services and resources from
This guide is designed for project teams that want to know MongoDB and the community that help build MongoDB
how to migrate from an RDBMS to MongoDB. We provide skills and proficiency, including free, web-based training,
a step-by-step roadmap, depicted in Figure 2. support, and consulting. See the MongoDB Services
section later in this guide for more detail.
Many links are provided throughout this document to help
guide users to the appropriate resources online. For the
most current and detailed information on particular topics,
please see the online documentation.

1
 Or
Organization
ganization Migrated F
Frrom Applic
Application
ation

eHarmony Oracle & Postgres Customer Data Management & Analytics

Shutterfly Oracle Web and Mobile Services

Cisco Multiple RDBMS Analytics, Social Networking

Craigslist MySQL Archive

Under Armour Microsoft SQL Server eCommerce

Foursquare PostgreSQL Social, Mobile Networking Platforms

MTV Networks Multiple RDBMS Centralized Content Management

Buzzfeed MySQL Real-Time Analytics

Verizon Oracle Single View, Employee Systems

The Weather Channel Oracle & MySQL Mobile Networking Platforms

Figur
Figuree 1: Case Studies

Schema Design From the legacy relational data model that flattens data
into rigid 2-dimensional tabular structures of rows and
columns
The most fundamental change in migrating from a
To a rich and dynamic document data model with
relational database to MongoDB is the way in which the
embedded sub-documents and arrays
data is modeled.

As with any data modeling exercise, each use case will be

different, but there are some general considerations that
you apply to most schema migration projects.

Before exploring schema design, Figure 3 provides a

useful reference for translating terminology from the
relational to MongoDB world.

Schema design requires a change in perspective for data

architects, developers, and DBAs:

Figur
Figure
e 2: Migration Roadmap

2
 R DB
DBMMS MongoDB embedded data structures. For example, data that belongs
to a parent-child relationship in two RDBMS tables would
Database Database commonly be collapsed (embedded) into a single
Table Collection document in MongoDB.

Row Document

Column Field

Index Index

JOIN Embedded document, document references,

or $lookup to combine data from different
collections

Figur
Figuree 3: Terminology Translation

From Rigid Tables to Flexible and Figur

Figuree 4: Relational Schema, Flat 2-D Tables
Dynamic BSON Documents
In Figure 4, the RDBMS uses the Pers_ID field to JOIN
Much of the data we use today has complex structures that the Person table with the Car table to enable the
can be modeled and represented more efficiently using
application to report each cars owner. Using the document
JSON (JavaScript Object Notation) documents, rather than
model, embedded sub-documents and arrays effectively
tables.
pre-JOIN data by combining related fields in a single data
MongoDB stores JSON documents in a binary structure. Rows and columns that were traditionally
representation called BSON (Binary JSON). BSON normalized and distributed across separate tables can now
encoding extends the popular JSON representation to be stored together in a single document, eliminating the
include additional data types such as int, decimal, long, and need to JOIN separate tables when the application has to
floating point. retrieve complete records.

With sub-documents and arrays, JSON documents also Modeling the same data in MongoDB enables us to create
align with the structure of objects at the application level. a schema in which we embed an array of sub-documents
This makes it easy for developers to map the data used in for each car directly within the Person document.
the application to its associated document in the database. {
first_name: Paul,
By contrast, trying to map the object representation of the surname: Miller,
data to the tabular representation of an RDBMS slows city: London,
location: [45.123,47.232],
down development. Adding Object Relational Mappers cars: [
(ORMs) can create additional complexity by reducing the { model: Bentley,
flexibility to evolve schemas and to optimize queries to year: 1973,
value: 100000, .},
meet new application requirements. { model: Rolls Royce,
year: 1965,
The project team should start the schema design process value: 330000, .},
by considering the applications requirements. It should ]
}
model the data in a way that takes advantage of the
document models flexibility. In schema migrations, it may
be easy to mirror the relational databases flat schema to
the document model. However, this approach negates the
advantages enabled by the document models rich,

3
 platform in Figure 5. In this example, the application relies
on the RDBMS to join five separate tables in order to build
the blog entry. With MongoDB, all of the blog data is
contained within a single document, linked with a single
reference to a user document that contains both blog and
comment authors.

Other Advantages of the Document Model

In addition to making it more natural to represent data at
the database level, the document model also provides
performance and scalability advantages:

The complete document can be accessed with a single

call to the database, rather than having to JOIN multiple
tables to respond to a query. The MongoDB document
is physically stored as a single object, requiring only a
single read from memory or disk. On the other hand,
RDBMS JOINs require multiple reads from multiple
physical locations.

As documents are self-contained, distributing the

database across multiple nodes (a process called
sharding) becomes simpler and makes it possible to
achieve massive horizontal scalability on commodity
hardware. The DBA no longer needs to worry about the
performance penalty of executing cross-node JOINs
(should they even be possible in the existing RDBMS)
to collect data from different tables.

Joining Collections
Typically it is most advantageous to take a denormalized
data modeling approach for operational databases the
efficiency of reading or writing an entire record in a single
Figur
Figuree 5: Pre-JOINing Data to Collapse 5 RDBMS Tables operation outweighing any modest increase in storage
to 2 BSON Documents requirements. However, there are examples where
normalizing data can be beneficial, especially when data
In this simple example, the relational model consists of only from multiple sources needs to be blended for analysis
two tables. (In reality most applications will need tens, this can be done using the $lookup stage in the
hundreds or even thousands of tables.) This approach does MongoDB Aggregation Framework.
not reflect the way architects think about data, nor the way
in which developers write applications. The document The Aggregation Framework is a pipeline for data
model enables data to be represented in a much more aggregation modeled on the concept of data processing
natural and intuitive way. pipelines. Documents enter a multi-stage pipeline that
transforms the documents into aggregated results. The
To further illustrate the differences between the relational pipeline consists of stages; each stage transforms the
and document models, consider the example of a blogging documents as they pass through.

4
While not offering as rich a set of join operations as some MongoDB
Applic
Application
ation R DB
DBMMS Action
RDBMSs, $lookup provides a left outer equi-join which Action
provides convenience for a selection of use cases. A left
Create INSERT to (n) insert() to 1
outer equi-join, matches and embeds documents from the Product tables (product document
"right" collection in documents from the "left" collection. Record description, price,
manufacturer, etc.)
As an example if the left collection contains order
documents from a shopping cart application then the Display SELECT and JOIN find() single
Product (n) product tables document
$lookup operator can match the product_id references
Record
from those documents to embed the matching product
details from the products collection. Add INSERT to review insert() to
Product table, foreign key review
MongoDB 3.4 introduces a new aggregation stage called Review to product record collection,
reference to
$graphLookup to recursively lookup a set of documents
product
with a specific defined relationship to a starting document. document
Developers can specify the maximum depth for the
More
recursion, and apply additional filters to only search nodes
Actions
that meet specific query predicates. $graphLookup can
recursively query within a single collection, or across Figur
Figuree 6: Analyzing Queries to Design the Optimum
Schema
multiple collections.

Worked examples of using $lookup as well as other This analysis helps to identify the ideal document schema
agggregation stages can be found in the blog Joins and and indexes for the application data and workload, based
Other Aggregation Enhancements. on the queries and operations to be performed against it.

The project team can also identify the existing application's

Defining the Document Schema most common queries by analyzing the logs maintained by
The applications data access patterns should drive schema the RDBMS. This analysis identifies the data that is most
design, with a specific focus on: frequently accessed together, and can therefore potentially
be stored together within a single MongoDB document. An
The read/write ratio of database operations and example of this process is documented in the Apollo
whether it is more important to optimize performance Groups migration from Oracle to MongoDB when
for one over the other developing a new cloud-based learning management
The types of queries and updates performed by the platform.
database

The life-cycle of the data and growth rate of documents Modeling Relationships with Embedding
As a first step, the project team should document the and Referencing
operations performed on the applications data, comparing: Deciding when to embed a document or instead create a
reference between separate documents in different
1. How these are currently implemented by the relational
collections is an application-specific consideration. There
database
are, however, some general considerations to guide the
2. How MongoDB could implement them decision during schema design.
Figure 6 represents an example of this exercise.

5
Embedding With m:1 or m:m relationships where embedding would
not provide sufficient read performance advantages to
Data with a 1:1 or 1:many relationship (where the many
outweigh the implications of data duplication
objects always appear with, or are viewed in the context of
their parent documents) are natural candidates for Where the object is referenced from many different
embedding within a single document. The concept of data sources
ownership and containment can also be modeled with To represent complex many-to-many relationships
embedding. Using the product data example above,
To model large, hierarchical data sets
product pricing both current and historical should be
embedded within the product document since it is owned The $lookup stage in an aggregation pipeline can be used
by and contained within that specific product. If the product to match the references with the _ids from the second
is deleted, the pricing becomes irrelevant. collection to automatically embed the referenced data in
the result set.
Architects should also embed fields that need to be
modified together atomically. (Refer to the Application
Integration section of this guide for more information.) Different Design Goals
Not all 1:1 and 1:m relationships should be embedded in a Comparing these two design options embedding
single document. Referencing between documents in sub-documents versus referencing between documents
different collections should be used when: highlights a fundamental difference between relational and
document databases:
A document is frequently read, but contains an
embedded document that is rarely accessed. An The RDBMS optimizes data storage efficiency (as it
example might be a customer record that embeds was conceived at a time when storage was the most
copies of the annual general report. Embedding the expensive component of the system.
report only increases the in-memory requirements (the
MongoDBs document model is optimized for how the
working set) of the collection
application accesses data (as performance, developer
One part of a document is frequently updated and time, and speed to market are now more important than
constantly growing in size, while the remainder of the storage volumes).
document is relatively static
Data modeling considerations, patterns and examples
The combined document size would exceed MongoDBs including embedded versus referenced relationships are
16MB document limit discussed in more detail in the documentation.

Referencing
Indexing
Referencing enables data normalization, and can give more
In any database, indexes are the single biggest tunable
flexibility than embedding. But the application will issue
performance factor and are therefore integral to schema
follow-up queries to resolve the reference, requiring
design.
additional round-trips to the server.
Indexes in MongoDB largely correspond to indexes in a
References are usually implemented by saving the _id
relational database. MongoDB uses B-Tree indexes, and
field1 of one document in the related document as a natively supports secondary indexes. As such, it will be
reference. A second query is then executed by the immediately familiar to those coming from a SQL
application to return the referenced data. background.
Referencing should be used: The type and frequency of the applications queries should
inform index selection. As with all databases, indexing does

1. A required unique field used as the primary key within a MongoDB document, either generated automatically by the driver or specified by the user.

6
not come free: it imposes overhead on writes and resource creating array indexes if the field contains an array, it
(disk and memory) usage. will be indexed as an array index.

TTL Indexes. In some cases data should expire

Index Types automatically. Time to Live (TTL) indexes allow the user
to specify a period of time after which the database will
MongoDB has a rich query model that provides flexibility in automatically delete the data. A common use of TTL
how data is accessed. By default, MongoDB creates an indexes is applications that maintain a rolling window of
index on the documents _id primary key field. history (e.g., most recent 100 days) for user actions
such as clickstreams.
All user-defined indexes are secondary indexes. Any field
can be used for a secondary index, including fields within Geospatial Indexes. MongoDB provides geospatial
sub-documents and arrays. indexes to optimize queries related to location within a
two-dimensional space, such as projection systems for
Index options for MongoDB include:
the earth. These indexes allow MongoDB to optimize
Compound Indexes. Using index intersection queries for documents that contain a polygon or points
MongoDB can use more than one index to satisfy a that are closest to a given point or line; that are within a
query. This capability is useful when running ad-hoc circle, rectangle or polygon; or that intersect a circle,
queries as data access patterns are typically not known rectangle or polygon.
in advance. Where a query that accesses data based on Sparse Indexes. Sparse indexes only contain entries
multiple predicates is known, it will be more performant for documents that contain the specified field. Because
to use Compound Indexes, which use a single index MongoDB allows the data model to vary from one
structure to maintain references to multiple fields. For document to another, it is common for some fields to be
example, consider an application that stores data about present only in a subset of all documents. Sparse
customers. The application may need to find customers indexes allow for smaller, more efficient indexes when
based on last name, first name, and state of residence. fields are not present in all documents.
With a compound index on last name, first name, and
Partial Indexes. Partial Indexes can be viewed as a
state of residence, queries could efficiently locate
more flexible evolution of Sparse Indexes, where the
people with all three of these values specified. An
DBA can specify an expression that will be checked to
additional benefit of a compound index is that any
determine whether a document should be included in a
leading field(s) within the index can be used, so fewer
particular index. e.g. for an "orders" collection, an index
indexes on single fields may be necessary: this
on state and delivery company might only be needed for
compound index would also optimize queries looking for
active orders and so the index could be made
customers by last name or last name and first name.
conditional on {orderState: "active"} thereby
Unique Indexes. By specifying an index as unique, reducing the impact to memory, storage, and write
MongoDB will reject inserts of new documents or performance while still optimizing searches over the
updates to existing documents which would have active orders.
resulted in duplicate values for the indexed field. If a
Hash Indexes. Hash indexes compute a hash of the
compound index is specified as unique, the combination
value of a field and index the hashed value. The primary
of values must be unique.
use of this index is to enable hash-based sharding, a
Array Indexes. For fields that contain an array, each simple and uniform distribution of documents across
array value is stored as a separate index entry. For shards.
example, documents that describe a product might
Text Sear
Searcch Indexes. MongoDB provides a specialized
include an array containing its main attributes. If there is
index for text search that uses advanced,
an index on the attributes field, each attribute is indexed
language-specific linguistic rules for stemming,
and queries on the attribute field can be optimized by
tokenization and stop words. Queries that use the text
this index. There is no special syntax required for

7
 Figur
Figure
e 7: Visual Query Profiling in MongoDB Ops Manager

search index return documents in relevance order. Each The number of index entries scanned
collection may have at most one text index but it may The number of documents read
include multiple fields.
How long the query took to resolve, reported in
MongoDBs storage engines all support all index types and milliseconds
the indexes can be created on any part of the JSON
Alternate query plans that were assessed but then
document including inside sub-documents and array
rejected
elements making them much more powerful than those
offered by RDBMSs. While it may not be necessary to shard the database at the
outset of the project, it is always good practice to assume
that future scalability will be necessary (e.g., due to data
Optimizing Performance With Indexes
growth or the popularity of the application). Defining index
MongoDBs query optimizer selects the index empirically by keys during the schema design phase also helps identify
occasionally running alternate query plans and selecting keys that can be used when implementing MongoDBs
the plan with the best response time. The query optimizer auto-sharding for application-transparent scale-out.
can be overridden using the cursor.hint() method.
MongoDB provides a range of logging and monitoring tools
As with a relational database, the DBA can review query to ensure collections are appropriately indexed and queries
plans and ensure common queries are serviced by are tuned. These can and should be used both in
well-defined indexes by using the explain() function development and in production.
which reports on:
The MongoDB Database Profiler is most commonly used
The number of documents returned during load testing and debugging, logging all database
operations or only those events whose duration exceeds a
Which index was used if any
configurable threshold (the default is 100ms). Profiling
Whether the query was covered, meaning no documents data is stored in a capped collection where it can easily be
needed to be read to return results searched for relevant events it is often easier to query
Whether an in-memory sort was performed, which this collection than parsing the log files.
indicates an index would be beneficial

8
Delivered as part of MongoDBs Ops Manager and Cloud Each customer may buy or subscribe to different
Manager platforms, the new Visual Query Profiler provides services from their vendor, each with their own sets of
a quick and convenient way for operations teams and contracts.
DBAs to analyze specific queries or query families. The
Modeling this real-world variance in the rigid,
Visual Query Profiler (as shown in Figure 7) displays how
two-dimensional schema of a relational database is
query and write latency varies over time making it simple
complex and convoluted. In MongoDB, supporting variance
to identify slower queries with common access patterns
between documents is a fundamental, seamless feature of
and characteristics, as well as identify any latency spikes.
BSON documents.
The visual query profiler will analyze data it collects to
MongoDBs flexible and dynamic schemas mean that
provide recommendations for new indexes that can be
schema development and ongoing evolution are
created to improve query performance. Once identified,
straightforward. For example, the developer and DBA
these new indexes need to be rolled out in the production
working on a new development project using a relational
system and Ops/Cloud Manager automates that process
database must first start by specifying the database
performing a rolling index build which avoids any impact to
schema, before any code is written. At a minimum this will
the application.
take days; it often takes weeks or months.
MongoDB Compass provides the ability to visualize explain
MongoDB enables developers to evolve the schema
plans, presenting key information on how a query
through an iterative and agile approach. Developers can
performed for example the number of documents
start writing code and persist the objects as they are
returned, execution time, index usage, and more. Each
created. And when they add more features, MongoDB will
stage of the execution pipeline is represented as a node in
continue to store the updated objects without the need for
a tree, making it simple to view explain plans from queries
performing costly ALTER TABLE operations or re-designing
distributed across multiple nodes.
the schema from scratch.

These benefits also extend to maintaining the application

Schema Evolution and the Impact on
in production. When using a relational database, an
Schema Design
application upgrade may require the DBA to add or modify
MongoDBs dynamic schema provides a major advantage fields in the database. These changes require planning
versus relational databases. across development, DBAs, and operations teams to
synchronize application and database upgrades, agreeing
Collections can be created without first defining their on when to schedule the necessary ALTER TABLE
structure, i.e., document fields and their data types.
operations.
Documents in a given collection need not all have the
same set of fields. One can change the structure of As MongoDB allows schemas to evolve dynamically, such
documents just by adding new fields or deleting existing operations requires upgrading just the application, with
ones. typically no action required for MongoDB. Evolving
applications is simple, and project teams can improve
Consider the example of a customer record:
agility and time to market.
Some customers will have multiple office locations and
At the point that the DBA or developer determines that
lines of business, and some will not.
some constraints should be enforced on the document
The number of contact methods within each customer structure, Document Validation rules can be added
can be different. further details are provided later in this guide.
The information stored for each of these contact
methods can vary. For instance, some may have public
social media feeds which could be useful to monitor,
and some will not.

9
Application Integration MongoDB offers an extensive array of advanced query
operators.

With the schema designed, the project can move towards

integrating the application with the database using MongoDB Aggregation Framework
MongoDB drivers and tools. Aggregating data within any database is an important
DBAs can also configure MongoDB to meet the capability and a strength of the RDBMS.
applications requirements for data consistency and Many NoSQL databases do not have aggregation
durability. Each of these areas are discussed below. capabilities. As a result, migrating to NoSQL databases has
traditionally forced developers to develop workarounds,
such as:
MongoDB Drivers and the API
1. Building aggregations within their application code,
Ease of use and developer productivity are two of
increasing complexity and compromising performance
MongoDBs core design goals.
2. Exporting data to Hadoop to run MapReduce jobs
One fundamental difference between a SQL-based
against the data. This also drastically increases
RDBMS and MongoDB is that the MongoDB interface is
complexity, duplicates data across multiple data stores
implemented as methods (or functions) within the API of a
and does not allow for real-time analytics
specific programming language, as opposed to a
completely separate text-based language like SQL. This, 3. If available, writing native MapReduce operations within
coupled with the affinity between MongoDBs BSON the NoSQL database itself
document model and the data structures used in modern MongoDB provides the Aggregation Framework natively
programming languages, makes application integration within the database, which delivers similar functionality to
simple. the GROUP BY, JOIN, and related SQL statements.
MongoDB has idiomatic drivers for the most popular
When using the Aggregation Framework, documents in a
languages, including eleven developed and supported by
collection pass through an aggregation pipeline, where
MongoDB (e.g., Java, Python, .NET, PHP) and more than
they are processed in stages. Expressions produce output
30 community-supported drivers.
documents based on calculations performed on the input
MongoDBs idiomatic drivers minimize onboarding time for documents. The accumulator expressions used in the
new developers and simplify application development. For $group stage maintain state (e.g., totals, maximums,

instance, Java developers can simply code against minimums, averages, standard deviations, and related data)
MongoDB natively in Java; likewise for Ruby developers, as documents progress through the pipeline.
PHP developers, and so on. The drivers are created by
Additionally, the Aggregation Framework can manipulate
development teams that are experts in their given language
and combine documents using projections, filters,
and know how programmers prefer to work within those
redaction, lookups (JOINs), and recursive graph lookups.
languages.
The SQL to Aggregation Mapping Chart shows a number
of examples demonstrating how queries in SQL are
Mapping SQL to MongoDB Syntax handled in MongoDBs Aggregation Framework. To enable
For developers familiar with SQL, it is useful to understand more complex analysis, MongoDB also provides native
how core SQL statements such as CREATE, ALTER, support for MapReduce operations in both sharded and
INSERT, SELECT, UPDATE, and DELETE map to the unsharded collections.
MongoDB API. The documentation includes a comparison
chart with examples to assist in the transition to MongoDB
Query Language structure and semantics. In addition,

10
 Figur
Figure
e 8: Uncover new insights with powerful visualizations generated from MongoDB

Business Intelligence Integration Converts the returned results into the tabular format
MongoDB Connector for BI expected by the BI tool, which can then visualize the
data based on user requirements
Driven by growing requirements for self-service analytics,
faster discovery and prediction based on real-time Additionally, a number of Business Intelligence (BI)
operational data, and the need to integrate multi-structured vendors have developed connectors to integrate MongoDB
and streaming data sets, BI and analytics platforms are one with their suites (without using SQL), alongside traditional
of the fastest growing software markets. relational databases. This integration provides reporting, ad
hoc analysis, and dashboarding, enabling visualization and
To address these requirements, modern application data analysis across multiple data sources. Integrations are
stored in MongoDB can be easily explored with available with tools from a range of vendors including
industry-standard SQL-based BI and analytics platforms. Actuate, Alteryx, Informatica, JasperSoft, Logi Analytics,
Using the BI Connector, analysts, data scientists and MicroStrategy, Pentaho, QlikTech, SAP Lumira, and Talend.
business users can now seamlessly visualize
semi-structured and unstructured data managed in
MongoDB, alongside traditional data in their SQL Atomicity in MongoDB
databases, using the same BI tools deployed within millions
Relational databases typically have well developed features
of enterprises.
for data integrity, including ACID transactions and
constraint enforcement. Rightly, users do not want to
SQL-based BI tools such as Tableau expect to connect to a sacrifice data integrity as they move to new types of
data source with a fixed schema presenting tabular data. databases. With MongoDB, users can maintain many
This presents a challenge when working with MongoDBs capabilities of relational databases, even though the
dynamic schema and rich, multi-dimensional documents. In technical implementation of those capabilities may be
order for BI tools to query MongoDB as a data source, the different.
BI Connector does the following:
MongoDB write operations are ACID at the document level
Provides the BI tool with the schema of the MongoDB including the ability to update embedded arrays and
collections to be visualized. Users can review the sub-documents atomically. By embedding related fields
schema output to ensure data types, sub-documents, within a single document, users get the same integrity
and arrays are correctly represented guarantees as a traditional RDBMS, which has to
Translates SQL statements issued by the BI tool into synchronize costly ACID operations and maintain
equivalent MongoDB queries that are then sent to referential integrity across separate tables.
MongoDB for processing

11
Document-level atomicity in MongoDB ensures complete logging, where users are typically analyzing trends in the
isolation as a document is updated; any errors cause the data, rather than discrete events.
operation to roll back and clients receive a consistent view
With stronger write concerns, write operations wait until
of the document.
MongoDB applies and acknowledges the operation. This is
Despite the power of single-document atomic operations, MongoDBs default configuration. The behavior can be
there may be cases that require multi-document further tightened by also opting to wait for replication of
transactions. There are multiple approaches to this the write to:
including using the findAndModify command that allows
A single secondary
a document to be updated atomically and returned in the
same round trip. findAndModify is a powerful primitive on A majority of secondaries
top of which users can build other more complex A specified number of secondaries
transaction protocols. For example, users frequently build
All of the secondaries even if they are deployed in
atomic soft-state locks, job queues, counters and state
different data centers (users should evaluate the
machines that can help coordinate more complex
impacts of network latency carefully in this scenario)
behaviors. Another alternative entails implementing a
two-phase commit to provide transaction-like semantics.
The documentation describes how to do this in MongoDB,
and important considerations for its use.

Maintaining Strong Consistency

By default, MongoDB directs all read operations to primary
servers, ensuring strong consistency. Also, by default any
reads from secondary servers within a MongoDB replica
set will be eventually consistent much like master / slave
replication in relational databases.

Administrators can configure the secondary replicas to

handle read traffic using MongoDBs Read Preferences,
which control how clients' read operations are routed to
members of a replica set.

Write Durability
MongoDB uses write concerns to control the level of write
guarantees for data durability. Configurable options extend
from simple fire and forget operations to waiting for
acknowledgments from multiple, globally distributed
Figur
Figuree 9: Configure Durability per Operation
replicas.

If opting for the most relaxed write concern, the application The write concern can also be used to guarantee that the
can send a write operation to MongoDB then continue change has been persisted to disk before it is
processing additional requests without waiting for a acknowledged.
response from the database, giving the maximum
The write concern is configured through the driver and is
performance. This option is useful for applications like
highly granular it can be set per-operation, per-collection,

12
or for the entire database. Users can learn more about unlike an RDBMS where everything must be defined and
write concerns in the documentation. enforced. For any key it might be appropriate to check:

MongoDB uses write-ahead logging to an on-disk journal That it exists

to guarantee write operation durability and to provide crash If it does exist, that the value is of the correct type
resilience.
That the value is in a particular format (regular
Before applying a change to the database whether it is a expressions can be used to check if the contents of the
write operation or an index modification MongoDB writes string matches a particular pattern that its a properly
the change operation to the journal. If a server failure formatted email address, for example)
occurs or MongoDB encounters an error before it can
That the value falls within a given range
write the changes from the journal to the database, the
journaled operation can be reapplied, thereby maintaining a As an example, assume that the following checks need to
consistent state when the server is recovered. be enforced on the contacts collection:

The year of birth is no later than 1994.

Implementing Validation & Constraints The document contains a phone number and/or an
email address.
Foreign Keys When present, the phone number and email addresses
are strings.
As demonstrated earlier, MongoDBs document model can
often eliminate the need for JOINs by embedding data That can be achieved by defining this Document Validation
within a single, atomically updated BSON document. This rule:
same data model can also reduce the need for Foreign Key
integrity constraints. db.runCommand({
collMod: "contacts",
validator: {
If you do want Foreign Key behavior from the database
$and: [
then third party frameworks such as Mongoose or {year_of_birth: {$lte: 1994}},
PyMODM can be used. {$or: [
{phone: { $type: "string" }},
{email: { $type: "string" }}
]}]
Document Validation }})

Dynamic schemas bring great agility, but it is also important Adding the validation checks to a collection is very intuitive
that controls can be implemented to maintain data quality, to any developer or DBA familiar with MongoDB, as
especially if the database is powering multiple applications, Document Validation uses the standard MongoDB Query
or is integrated into a larger data management platform Language.
that feeds into upstream and downstream systems. Rather
than delegating enforcement of these controls back into Validation rules can be managed from the Compass GUI.
application code, MongoDB provides Document Validation Rules can be created and modified directly using a simple
within the database. Users can enforce checks on point and click interface, and any documents violating the
document structure, data types, data ranges, and the rules can be clearly presented. DBAs can then use
presence of mandatory fields. As a result, DBAs can apply Compasss CRUD support to fix data quality issues in
data governance standards, while developers maintain the individual documents.
benefits of a flexible document model.

There is significant flexibility to customize which parts of Enforcing Constraints With Indexes
the documents are and ar
aree not validated for any collection As discussed in the Schema Design section, MongoDB
supports unique indexes natively, which detect and raise an

13
error to any insert operation that attempts to load a
duplicate value into a collection. A tutorial is available that
describes how to create unique indexes and eliminate
duplicate entries from existing collections.

Views
DBAs can define non-materialized views that expose only a
subset of data from an underlying collection, i.e. a view that
filters out specific fields. DBAs can define a view of a
collection that's generated from an aggregation over
another collection(s) or view. Permissions granted against
the view are specified separately from permissions granted
to the underlying collection(s).
Figur
Figuree1
10:
0: Multiple Options for Data Migration
Views are defined using the standard MongoDB Query
Language and aggregation pipeline. They allow the As records are retrieved from the RDBMS, the
inclusion or exclusion of fields, masking of field values, application writes them back out to MongoDB in the
filtering, schema transformation, grouping, sorting, limiting, required document schema.
and joining of data using $lookup and $graphLookup to Consistency checkers, for example using MD5
another collection. checksums, can be used to validate the migrated data.

You can learn more about MongoDB read-only views from All newly created or updated data is written to
the documentation. MongoDB only.

Shutterfly used this incremental approach to migrate the

Migrating Data to MongoDB metadata of 6 billion images and 20TB of data from Oracle
to MongoDB.

Project teams have multiple options for importing data from Incremental migration can be used when new application
existing relational databases to MongoDB. The tool of features are implemented with MongoDB, or where
choice should depend on the stage of the project and the multiple applications are running against the legacy
existing environment. RDBMS. Migrating only those applications that are being
modernized enables teams to divide projects into more
Many users create their own scripts, which transform
manageable and agile development sprints.
source data into a hierarchical JSON structure that can be
imported into MongoDB using the mongoimport tool. Incremental migration eliminates disruption to service
availability while also providing fail-back should it be
Extract Transform Load (ETL) tools are also commonly
necessary to revert back to the legacy database.
used when migrating data from relational databases to
MongoDB. A number of ETL vendors including Informatica, Many organizations create feeds from their source
Pentaho, and Talend have developed MongoDB connectors systems, dumping daily updates from an existing RDBMS
that enable a workflow in which data is extracted from the to MongoDB to run parallel operations, or to perform
source database, transformed into the target MongoDB application development and load testing. When using this
schema, staged, and then loaded into collections. approach, it is important to consider how to handle deletes
to data in the source system. One solution is to create A
Many migrations involve running the existing RDBMS in
and B target databases in MongoDB, and then alternate
parallel with the new MongoDB database, incrementally
daily feeds between them. In this scenario, Database A
transferring production data:

14
receives one daily feed, then the application switches the
MongoDB Atlas: Database as a
next day of feeds to Database B. Meanwhile the existing
Database A is dropped, so when the next feeds are made Service For MongoDB
to Database A, a whole new instance of the source
database is created, ensuring synchronization of deletions
MongoDB can run the database for you! MongoDB Atlas
to the source data.
provides all of the features of MongoDB, without the
operational heavy lifting required for any new application.
MongoDB Atlas is available on-demand through a
Operational Agility at Scale pay-as-you-go model and billed on an hourly basis, letting
you focus on what you do best.
The considerations discussed thus far fall into the domain
Its easy to get started use a simple GUI to select the
of the data architects, developers, and DBAs. However, no
instance size, region, and features you need. MongoDB
matter how elegant the data model or how efficient the
Atlas provides:
indexes, none of this matters if the database fails to
perform reliably at scale or cannot be managed efficiently. Security features to protect access to your data

The final set of considerations in migration planning should Built in replication for always-on availability, tolerating
focus on operational issues. complete data center failure

Backups and point in time recovery to protect against

The MongoDB Operations Best Practices guide is the
data corruption
definitive reference to learn more on this key area.
Fine-grained monitoring to let you know when to scale.
The Operations Guide discusses:
Additional instances can be provisioned with the push
Management, monitoring, and backup with MongoDB of a button
Ops Manager or MongoDB Cloud Manager, which is the Automated patching and one-click upgrades for new
best way to run MongoDB within your own data center major versions of the database, enabling you to take
or public cloud, along with tools such as mongotop, advantage of the latest and greatest MongoDB features
mongostat, and mongodump
A choice of cloud providers, regions, and billing options
High availability with MongoDB Replica Sets, providing
MongoDB Atlas is versatile. Its great for everything from a
self-healing recovery from failures and supporting
quick Proof of Concept, to test/QA environments, to
scheduled maintenance with no downtime
complete production clusters. If you decide you want to
Scalability using MongoDB auto-sharding (partitioning) bring operations back under your control, it is easy to move
across clusters of commodity servers, with application your databases onto your own infrastructure and manage
transparency them using MongoDB Ops Manager or MongoDB Cloud
Hardware selection with optimal configurations for Manager. The user experience across MongoDB Atlas,
memory, disk and CPU Cloud Manager, and Ops Manager is consistent, ensuring
that disruption is minimal if you decide to migrate to your
Security including LDAP, Kerberos and x.509
own infrastructure.
authentication, field-level access controls, user-defined
roles, auditing, encryption of data in-flight and at-rest, MongoDB Atlas is automated, its easy, and its from the
and defense-in-depth strategies to protect the database creators of MongoDB. Learn more and take it for a spin.

15
MongoDB Stitch: Backend as a MongoDB University
Service Courses are available for both developers and DBAs:

Free, web-based classes, delivered over 7 weeks,

MongoDB Stitch is a backend as a service (BaaS), giving supported by lectures, homework and forums to interact
developers a REST-like API to MongoDB, and with instructors and other students. Over 350,000
composability with other services, backed by a robust students have already enrolled in these classes.
system for configuring fine-grained data access controls.
Public training events held at MongoDB facilities.
Stitch provides native SDKs for JavaScript, iOS, and
Android. Private training customized to an organizations
specific requirements, delivered at their site.
Built-in integrations give your application frontend access
to your favorite third party services: Twilio, AWS S3, Slack, Learn more.
Mailgun, PubNub, Google, and more. For ultimate flexibility,
you can add custom integrations using MongoDB Stitch's
Community Resources and Consulting
HTTP service.
In addition to training, there is a range of other resources
MongoDB Stitch allows you to compose multi-stage and services that project teams can leverage:
pipelines that orchestrate data across multiple services;
where each stage acts on the data before passing its Tec
echnic
hnical
al rresour
esources
ces for the community are available
results on to the next. through forums on Google Groups, StackOverflow and
IRC
Unlike other BaaS offerings, MongoDB Stitch works with
your existing as well as new MongoDB clusters, giving you Consulting pac
packkages include health checks, custom
access to the full power and scalability of the database. By consulting and access to dedicated Technical Account
defining appropriate data access rules, you can selectively Managers. More details are available from the
expose your existing MongoDB data to other applications MongoDB Consulting page.
through MongoDB Stitch's API.

Take advantage of the free tier to get started; when you Conclusion
need more bandwidth, the usage-based pricing model
ensures you only pay for what you consume. Learn more
Following the best practices outlined in this guide can help
and try it out for yourself.
project teams reduce the time and risk of database
migrations, while enabling them to take advantage of the

Supporting Your Migration: benefits of MongoDB and the document model. In doing
so, they can quickly start to realize a more agile, scalable
MongoDB Services and cost-effective infrastructure, innovating on applications
that were never before possible.
MongoDB and the community offer a range of resources
and services to support migrations by helping users build
MongoDB skills and proficiency. MongoDB services
We Can Help
include training, support, forums, and consulting. Refer to
the "We Can Help" section below to learn more about We are the MongoDB experts. Over 3,000 organizations
support from development through to production. rely on our commercial products, including startups and
more than half of the Fortune 100. We offer software and
services to make your life easier:

16
MongoDB Enterprise Advanced is the best way to run
Resources
MongoDB in your data center. It's a finely-tuned package
of advanced software, support, certifications, and other
services designed for the way you do business. For more information, please visit mongodb.com or contact
us at sales@mongodb.com.
MongoDB Atlas is a database as a service for MongoDB,
letting you focus on apps instead of ops. With MongoDB Case Studies (mongodb.com/customers)
Atlas, you only pay for what you use with a convenient Presentations (mongodb.com/presentations)
hourly billing model. With the click of a button, you can Free Online Training (university.mongodb.com)
scale up and down when you need to, with no downtime, Webinars and Events (mongodb.com/events)
full security, and high performance. Documentation (docs.mongodb.com)
MongoDB Enterprise Download (mongodb.com/download)
MongoDB Stitch is a backend as a service (BaaS), giving
MongoDB Atlas database as a service for MongoDB
developers full access to MongoDB, declarative read/write
(mongodb.com/cloud)
controls, and integration with their choice of services.
MongoDB Stitch backend as a service (mongodb.com/
MongoDB Cloud Manager is a cloud-based tool that helps cloud/stitch)
you manage MongoDB on your own infrastructure. With
automated provisioning, fine-grained monitoring, and
continuous backups, you get a full management suite that
reduces operational overhead, while maintaining full control
over your databases.

MongoDB Professional helps you manage your

deployment and keep it running smoothly. It includes
support from MongoDB engineers, as well as access to
MongoDB Cloud Manager.

Development Support helps you get up and running quickly.

It gives you a complete package of software and services
for the early stages of your project.

MongoDB Consulting packages get you to production

faster, help you tune performance in production, help you
scale, and free you up to focus on your next release.

MongoDB Training helps you become a MongoDB expert,

from design to operating mission-critical systems at scale.
Whether you're a developer, DBA, or architect, we can
make you better at MongoDB.

New York Palo Alto Washington, D.C. London Dublin Barcelona Sydney Tel Aviv
US 866-237-8815 INTL +1-650-440-4474 info@mongodb.com
2017 MongoDB, Inc. All rights reserved.

17