Automating MongoDB with SaltStack

Database deployment for a multiple number of servers becomes more complex and time consuming with time when adding new resources or making changes. In addition, there is a likelihood of human errors that may lead to catastrophic outcomes whenever the system is configured manually.  

A database deployment automation tool will enable us to deploy a database across multiple servers ranging from development to production environments. The results from an automated deployment are reliable, more efficient and predictable besides providing the current state information of your nodes which can be further used to plan for resources you will need to add to your servers. With a well-managed deployment, the productivity of both development and operational teams improves thereby enabling the business to develop faster, accomplish more and due to easy frequent deployment, the overall software setup will be ultimately better and function reliably for end-users. 

MongoDB can be deployed manually but the task becomes more and more cumbersome when you have to configure a cluster of many members being hosted on different servers. We therefore need to resolve to use an automotive tool that can save us the stress. Some of the available tools that can be used include Puppet, Chef, Ansible, and SaltStack.

The main benefits of deploying your MongoDB with any of these tools are:

1. Time saving. Imagine having 50 nodes for your database and you need to update MongoDB version for each. This will take you ages going through the process. However, with an automatic tool, you will just need to write some instructions and issues a command to do the rest of the update for you. Developers will then have time to work on new features rather than fixing manual deployments.
2. Reduced errors hence customer satisfaction. Making new updates may introduce errors to a database system especially if the configuration has to be done manually. With a tool like SaltStack, removing manual steps reduces human error and frequent updates with new features will address customer needs hence keeping the organization competitive.
3. Lower configuration cost. With a deployment tool, anyone can deploy even yourself since the process itself will be much easier. This will eliminate the need for experts to do the work and reduced errors

What is SaltStack

SaltStack is an open-source remote execution tool and a configuration management system developed in Python. 

The remote execution features are used to run commands on various machines in parallel with a flexible targeting system. If for example you have 3 server machines and you would like to install MongoDB for each, you can run the installation commands on these machines simultaneously from a master node. 

In terms of configuration management, a client-server interface is established to ease and securely transform the infrastructure components into the desired state.

SaltStack Architecture

The basic setup model for SaltStack is Client-Server where the server can be referred to as the master and the Clients as slaves. The master issues command or rather instructions as the controlling system that need to be executed by the clients/minions which are the controlled systems.

SaltSack Components

The following are what SaltStack is made of

1. Master: Responsible for issuing instructions to the slaves and change them to the desired state after execution.
2. Minion: It is the controlled system which needs to be transformed into some desired state.
3. Salt Grains:  this is static data or metadata regarding the minion and it constitutes information like the model, serial number, memory capacity, and the Operating System. They are collected when the minion first connects to the server. They can be used for targeting a certain group of minions in relation to some aspect. For example, you can run a command stating, install MongoDB for all machines with a Windows operating system. 
4. Execution Modules/instructions: These are Ad hoc commands issued to one or more target minions and are executed from the command line.
5. Pillars: are user defined variables distributed among the minions. They are used for: minion configuration, highly sensitive data, arbitrary data, and variables. Not all minions are accessible to all pillars, one can restrict which pillars are for a certain group of minions.
6. State files. This is the core of Salt state System (SLS) and it represents the state in which the system should be in. It is an equivalent to a playbook in case of Ansible considering that they are also in YAML format i.e

#/srv/salt/mongodbInstall.sls (file root)

install_mongodb: (task id)

pkg.installed: (state declaration)

-name:mongodb  (name of package to install)

7. Top file: Used to map a group of machines and define which state files should be applied . i.e.

#/srv/salt/top.sls

  base:

   ‘minion1’:

     -mongodb

8. Salt Proxy:  This is a feature that enables controlling devices that cannot run a standard salt-minion. They include network gears with an API running on a proprietary OS, devices with CPU and memory limitations or ones that cannot run minions due to security reasons. A Junos proxy has to be used for discovery, control, remote execution and state management of these devices.

SaltStack Installation

We can use the pip command to install SaltStack as 

$ pip install salt

To confirm the installation, run the command $ salt –version and you should get something like salt 2019.2.2 (Fluorine)

Before connecting to the master the minion will require a minimum configuration of the master ip address and minion id which will be used by the master for its reference. These configurations can be done in the files /etc/salt/minion.

We can then run the master in various modes that is daemon or in debug mode. For the daemon case you will have $salt-master -d and for debug mode,  $salt-master -l debug. You will need to accept the minion’s key before starting it by running $ salt-key -a nameOfMinion. To list the available keys, run $ salt-key -l

In the case of the minion, we can start it with $salt-minion -l debug.

For example, if we want to create a file in all the minions from the master, we can run the command 

$ salt ‘’*” file.touch ‘/tmp/salt_files/sample.text

All nodes will have a new sample.text file in the salt_files folder. The * option is used to refer to all minions. To specify for example all minions with id name having the string minion, we will use a regex expression as below 

$ salt “minion*” file.touch ‘/tmp/salt_files/sample.text

To see the metadata collected for a given minion, run:

$salt ‘minion1’ grains.items.

Setting up MongoDB with SaltStack

We can create a database called myAppdata with the setDatabase.sls with the contents below 

classes:

- service.mongodb.server.cluster

parameters:

   _param:

     mongodb_server_replica_set: myAppdata

     mongodb_myAppdata_password: myAppdataPasword

     mongodb_admin_password: cloudlab

     mongodb_shared_key: xxx

   mongodb:

     server:

       database:

         myAppdata:

           enabled: true

           password: ${_param:mongodb_myAppdata_password}

           users:

           -  name: myAppdata

              password: ${_param:mongodb_myAppdata_password}

Starting a Single MongoDB Server 

mongodb:

  server:

    enabled: true

    bind:

      address: 0.0.0.0

      port: 27017

    admin:

      username: admin

      password: myAppdataPasword

    database:

      myAppdata:

        enabled: true

        encoding: 'utf8'

        users:

        - name: 'username'

          password: 'password'

Setting up a MongoDB Cluster with SaltStack

mongodb:

  server:

    enabled: true

    logging:

      verbose: false

      logLevel: 1

      oplogLevel: 0

    admin:

      user: admin

      password: myAppdataPasword

    master: mongo01

    members:

      - host: 192.168.100.11

        priority: 2

      - host: 192.168.101.12

      - host: 192.168.48.13

    replica_set: default

    shared_key: myAppdataPasword

Conclusion

Like ClusterControl, SaltStack is an automation tool that can be used to ease deployment and operations tasks. With an automation tool, there are reduced errors, reduced time of configuration, and more reliable results.