According to Wikipedia, a ceilometer is a device that uses a laser or other light source to determine the height of a cloud base. And it is also the name of the framework for monitoring and metering OpenStack. It collects measurements within OpenStack so that no two agents would need to be written to collect the same data. 

Ceilometer collects metering data (CPU, network costs) that can be used for billing purposes with Openstack. Because it requires a lot of writes, the preferred database is MongoDB. Although there are drivers for other database backends, the storage driver for MongoDB is considered feature-complete at this time and therefore recommended for production usage.

In this post we are going to show you how to deploy a minimal MongoDB replica set (two MongoDB instances) and install Ceilometer services on OpenStack’s controller and compute nodes. This example assumes that we already have an OpenStack controller and a compute node running on Ubuntu Precise with three additional nodes specifically for MongoDB and ClusterControl.

Our architecture is illustrated as follows:

Our hosts definitions on every hosts are as below:	controller	compute1	clustercontrol	mongo1	mongo2


Deploying MongoDB Replica Set

1. We will use the MongoDB Configurator to deploy a Replica Set consisting of two MongoDB instances. The following options have been used for the deployment package:

Configuration        : Single Replicaset

Vendor               : 10gen

Number of mongod in each replica set: 2 mongod

OS user              : Ubuntu

Use ‘smallfiles’     : yes

ClusterControl server:

Mongo Servers        : 1)



2. Download the deployment package into the ClusterControl node and start the deployment:

$ wget
$ tar -xzf s9s-mongodb-10gen-1.0.0.tar.gz
$ cd s9s-mongodb-10gen-1.0.0/mongodb/scripts/install/
$ bash ./ 2>&1 | tee cc.log

3. Once the deployment is complete, register the MongoDB Replica Set with ClusterControl UI. Open and enter the API token and CMONAPI URL. Once registered, you should see a mongodb database appears in the UI dashboard page, similar to screenshot below:

The database platform is ready and we can now proceed to install Ceilometer services.

Note: You might also consider to add a MongoDB arbiter to your Replica Set to maintain reliable quorum. For instance, to deploy an arbiter on the ClusterControl Server, make sure to install MongoDB server packages beforehand and use following command to add the arbiter into the current replica set from the ClusterControl node:

$ sudo s9s_mongodb_admin --add-arbiter -i 1 -N my_mongodb_0 -h -P 30000 -d /var/lib/mongodb

Once done, the arbiter node should listed under Nodes page:


Installing Ceilometer

Controller Node

1. Add MongoDB repository into /etc/apt/sources.list:

$ echo “deb dist 10gen” | sudo tee -a /etc/apt/sources.list

2. Install Ceilometer services and mongodb-clients on the OpenStack controller node:

$ sudo apt-get install ceilometer-api ceilometer-collector ceilometer-agent-central python-ceilometerclient mongodb-clients

3. Edit the connection string to use mongodb in /etc/ceilometer/ceilometer.conf under [database] section. Make sure you specify the PRIMARY mongod instance (as in this case is mongo1):


4. Generate a shared secret key:

$ openssl rand -hex 10

And replace the value for metering_secret under [publisher_rpc] section in /etc/ceilometer/ceilometer.conf:


5. Create a ceilometer user to authenticate with Keystone. Use the service tenant and assign user with admin role:

$ keystone user-create --name=ceilometer --pass=ceilometerpassword
$ keystone user-role-add --user=ceilometer --tenant=service --role=admin

6. Define the Keystone authentication token at /etc/ceilometer/ceilometer.conf by adding the following lines under the [keystone_authtoken] section:

auth_host = controller
auth_uri = http://controller:35357/v2.0
auth_protocol = http
admin_tenant_name = service
admin_user = ceilometer
admin_password = ceilometerpassword

7. Register the ceilometer service with Keystone and take note of the id generated as we will need it in the next step:

$ keystone service-create --name=ceilometer --type=metering --description="Ceilometer Metering Service"
|   Property  |              Value               |
| description |   Ceilometer Metering Service    |
|      id     | 8a6d406fe08042d5a1ac5793dc46fc69 |
|     name    |            ceilometer            |
|     type    |             metering             |

8. Create the endpoint for ceilometer services. Use the --service-id value generated in the previous command: 

$ keystone endpoint-create \
  --service-id=8a6d406fe08042d5a1ac5793dc46fc69 \
  --publicurl=http://controller:8777/ \
  --internalurl=http://controller:8777/ \

9. Let’s create a mongodb database and user for ceilometer. Firstly connect to mongo1 using the mongo client:

$ mongo --host mongo1

And execute following queries:

my_mongodb_0:PRIMARY> use ceilometer
my_mongodb_0:PRIMARY> db.addUser ( { user: "ceilometer", pwd: "ceilometerpassword" , roles: [ "readWrite", "dbAdmin" ] } )

10. Restart ceilometer services:

$ sudo service ceilometer-agent-central restart
$ sudo service ceilometer-api restart
$ sudo service ceilometer-collector restart

Compute Node

1. In this example, we just have a compute node used to run instances. So we are going to install the ceilometer agent package for nova compute. To add other service agents for metering services, please refer to Havana documentation:

$ sudo apt-get install ceilometer-agent-compute

2. Configure Nova with metering services by adding the following lines into /etc/nova/nova.conf under [default] section:


3. Add the same shared secret key into /etc/ceilometer/ceilometer.conf:


4. Restart ceilometer-agent and nova-compute: 

$ sudo service ceilometer-agent-compute restart
$ sudo service nova-compute restart


Log into the Horizon dashboard and you should able to see ceilometer has been enabled under System Info:


After a few minutes, verify that the Ceilometer services have started to insert data into the ceilometer database. The following summary can be retrieved from ClusterControl’s Overview page:


Deploying Ceilometer with MongoDB Sharded Cluster

As your OpenStack infrastructure expands, expect metering collections and your MongoDB dataset to grow. You should then look into moving your Replica Set to a Sharded Cluster topology. There are several ways to achieve this, either convert the Replica Set to a Sharded Cluster or manually export the data set to a new MongoDB Sharded Cluster. You can easily deploy a Sharded Cluster using the MongoDB Configurator.


Related Post