An Overview of MongoDB Atlas: Part One

The cloud computing approach addresses some of the challenges associated with running data processing systems. Data-driven companies are pushing out rapid business transformation with cloud services, and many see cloud services as a substantial enhancement in automation, reliability, and on-demand scaling than the traditional infrastructure models which came before. The on-demand nature of the Software-as-a-Service (SaaS) paradigm means organizations can buy what they need, when they need it. Of course, the cost and cost-effective aspects are crucial, but not the only ones.

In the design on system architectures, we are always looking for the systems which fits the right number of users, at the right level of performance for each. We want to avoid performance issues & bottlenecks, and if those issues happen, we want a system which adapts to the changing demand. 

We also want things faster. The agile development process is getting more and more popular; mainly because it accelerates the delivery of initial business value and (through a process of continuous planning and feedback) it can ensure that the ROI is maximized. 

Lastly, we want a reduction in complexity. A key feature of MongoDB is its built-in redundancy. If you have two or more data nodes, they can be configured as a replica set or mongodb shards. Without proper automation in place, it can be a recurring task for several teams (network, storage, OS, etc.). Cloud automation can help you to reduce dependencies between the various groups in your organization. For example, you may not need to involve the network team when you create a new database system. 

Cloud automation not only saves time and money but also make your organization more competitive in the challenging market. 

In this blog, we will take a look at Atlas, the solution from MongoDB that tries to address all of these problems.

Getting Started with MongoDB Atlas

To start with MongoDB Atlas go to https://cloud.mongodb.com. In the registration form, you need to provide bare minimum information like email, company, country, and mobile number. 

MongoDB Atlas does an excellent job in infrastructure provisioning, setup. The whole process uses a dynamic web interface that walks you through various deployment options. It’s easy, intuitive and doesn’t require specialized knowledge.

After the first login, you will be asked to build your first cluster in one of the three most significant clouds. Atlas works with Amazon AWS, Google Cloud, and Microsoft Azure.  Based on your choice, you can pick up the location of the preferred data center location. To increase availability, you can set Multi-Region, Workload Isolation, or set various Replication options. Each Atlas project supports up to 25 clusters, but after the contact with the support, you should be able to host more.

You need to select the appropriate size of the server, coupled with IO and storage capacity. In this article, we will use the free version. It is free to start with MongoDB Atlas for prototyping, early development or to learn. The credit card is not needed, so you don’t need to bother about hidden costs. The free edition called M0 Sandbox is limited to:

• 512MB storage
• vCPU shared
• RAM shared
• 100 max connections
• There is a limit of one M0 cluster per project.

For dedicated clusters, MongoDB Atlas is billed hourly based on how much you use. The rate depends on a number of factors, most importantly, the size and number of servers you use. The price starts with 0.08/hr (M10, 2GB RAM, 10GB storage, 1vCPU) to M700 with 768GB RAM, 4096 GB storage, 96vCPUs from $33.26/hr.  Obviously, you would need to include other cost factors like, for example, the cost of backups.

According to MongoDB calculations, an AWS a 3-node replica set of M40s and run it 24/7 for one month using the included 80GB of standard block storage would cost you around $947.

The basic setup works with replication. If you need sharding M30 instance type is a minimum (8GB RAM, 40GB storage, 2vCPUs, price from $0.54/hr). 

MongoDB Atlas Network Access Initial Setup

One of the first steps we need to do after the cluster creation is to enable an IP whitelist. To enable access from everywhere you can set whitelist entry to 0.0.0.0/0 but it’s not recommended. If you don’t know your IP address Atlas will help you to identify it. 

To keep your connection more secure you can also set up a network peering connection. This feature is not available for M0, M2, and M5 clusters. Network peering allows connectivity between MongoDB VPC and your cloud provider. Peer VPC network allows different VOC ti to communicate in private space, traffic doesn’t traverse the public internet. 

To start working with your new cluster create an initial user. Do it in the Database Access tab. MongoDB uses Salted Challenge Response Authentication Mechanism. It’s a security mechanism based on SHA-256, user credentials against the user’s name, password and authentication database. 

Migration of Existing MongoDB Cluster to MongoDB Atlas

There is also a possibility to migrate your existing on-prem cluster to Mongo Atlas. It’s done via a dedicated service called Live Migration Service. Atlas Live Migration process streams data through a MongoDB-controlled application server. 

Live migration works by keeping a cluster in MongoDB Atlas in sync with your source database. During this process, your application can continue to read and write from your source database. Since the process watches upcoming changes, all will be replicated, and migration can be done online. You decide when to change the application connection setting and do cutover. To do the process less prone Atlas provides Validate option which checks whitelist IP access, SSL configuration, CA, etc.

What’s important here is the service is free of charge.

If you don’t need online migration, you can also use mongoimport. Use mongo shell with minimum version 3.2.7 always use SSL. You can get test data from here.

​mongoimport --host TestCluster-shard-0/testcluster-shard-*****.azure.mongodb.net:27017,testcluster-shard-****.azure.mongodb.net:27017,testcluster-shard-******.azure.mongodb.net:27017 --ssl --username admin --authenticationDatabase admin  --type JSON --file city_inspections.json

2019-08-15T21:53:09.921+0200 WARNING: ignoring unsupported URI parameter 'replicaset'

2019-08-15T21:53:09.922+0200 no collection specified

2019-08-15T21:53:09.922+0200 using filename 'city_inspections' as collection

Enter password:



2019-08-15T21:53:14.288+0200 connected to: mongodb://testcluster-shard-*****.azure.mongodb.net:27017,testcluster-shard-*****.azure.mongodb.net:27017,testcluster-shard-*****.azure.mongodb.net:27017/?replicaSet=TestCluster-shard-0

2019-08-15T21:53:17.289+0200 [........................] test.city_inspections 589KB/23.2MB (2.5%)

2019-08-15T21:53:20.290+0200 [#.......................] test.city_inspections 1.43MB/23.2MB (6.2%)

2019-08-15T21:53:23.292+0200 [##......................] test.city_inspections 2.01MB/23.2MB (8.6%)

...

2019-08-15T21:55:09.140+0200 [########################] test.city_inspections 23.2MB/23.2MB (100.0%)

2019-08-15T21:55:09.140+0200 81047 document(s) imported successfully. 0 document(s) failed to import.

To check data, login with mongo shell.

mongo "mongodb+srv://testcluster-*****.azure.mongodb.net/test" --username admin

MongoDB shell version v4.2.0

Enter password:

connecting to: mongodb://testcluster-shard-00-00-*****.azure.mongodb.net:27017,testcluster-shard-00-02-*****.azure.mongodb.net:27017,testcluster-shard-00-01-*****.azure.mongodb.net:27017/test?authSource=admin&compressors=disabled&gssapiServiceName=mongodb&replicaSet=TestCluster-shard-0&ssl=true

2019-08-15T22:15:58.068+0200 I  NETWORK [js] Starting new replica set monitor for TestCluster-shard-0/testcluster-shard-00-00-*****.azure.mongodb.net:27017,testcluster-shard-00-02-*****.azure.mongodb.net:27017,testcluster-shard-00-01-*****.azure.mongodb.net:27017

2019-08-15T22:15:58.069+0200 I  CONNPOOL [ReplicaSetMonitor-TaskExecutor] Connecting to testcluster-shard-00-01-*****.azure.mongodb.net:27017

2019-08-15T22:15:58.070+0200 I  CONNPOOL [ReplicaSetMonitor-TaskExecutor] Connecting to testcluster-shard-00-00-*****.azure.mongodb.net:27017

2019-08-15T22:15:58.070+0200 I  CONNPOOL [ReplicaSetMonitor-TaskExecutor] Connecting to testcluster-shard-00-02-*****.azure.mongodb.net:27017

2019-08-15T22:15:58.801+0200 I  NETWORK [ReplicaSetMonitor-TaskExecutor] Confirmed replica set for TestCluster-shard-0 is TestCluster-shard-0/testcluster-shard-00-00-*****.azure.mongodb.net:27017,testcluster-shard-00-01-*****.azure.mongodb.net:27017,testcluster-shard-00-02-*****.azure.mongodb.net:27017

Implicit session: session { "id" : UUID("6a5d1ee6-064b-4ba8-881a-71aa4aef4983") }

MongoDB server version: 4.0.12

WARNING: shell and server versions do not match

MongoDB Enterprise TestCluster-shard-0:PRIMARY> show collections;

city_inspections

MongoDB Enterprise TestCluster-shard-0:PRIMARY> db.city_inspections.find();

{ "_id" : ObjectId("56d61033a378eccde8a83557"), "id" : "10284-2015-ENFO", "certificate_number" : 9287088, "business_name" : "VYACHESLAV KANDZHANOV", "date" : "Feb 25 2015", "result" : "No Violation Issued", "sector" : "Misc Non-Food Retail - 817", "address" : { "city" : "NEW YORK", "zip" : 10030, "street" : "FREDRCK D BLVD", "number" : 2655 } }

{ "_id" : ObjectId("56d61033a378eccde8a83559"), "id" : "10302-2015-ENFO", "certificate_number" : 9287089, "business_name" : "NYC CANDY STORE SHOP CORP", "date" : "Feb 25 2015", "result" : "No Violation Issued", "sector" : "Cigarette Retail Dealer - 127", "address" : { "city" : "NEW YORK", "zip" : 10030, "street" : "FREDRCK D BLVD", "number" : 2653 } }

...

{ "_id" : ObjectId("56d61033a378eccde8a8355e"), "id" : "10391-2015-ENFO", "certificate_number" : 3019415, "business_name" : "WILFREDO DELIVERY SERVICE INC", "date" : "Feb 26 2015", "result" : "Fail", "sector" : "Fuel Oil Dealer - 814", "address" : { "city" : "WADING RIVER", "zip" : 11792, "street" : "WADING RIVER MANOR RD", "number" : 1607 } }

Type "it" for more

MongoDB Enterprise TestCluster-shard-0:PRIMARY>

Conclusion

That’s all for part one. In the next article, we are going to cover monitoring, backups, day to day administration and MongoDB’s new service for building Data Lakes. Stay tuned!