Introducing the ClusterControl Alerting Integrations

The time to respond to outages is one of the most important metrics to measure, when you have a 24/7 business to run. Incident response applications, like VictorOps and PagerDuty, will report back their effectiveness using this metric. Next to responding to an outage, it is equally important to receive correct information in a timely way. If the incident is serious enough, the outage could potentially cost you a lot of money in lost business or simply because there was no availability.

Databases are becoming the key element of many applications and services, so any outage of a database could create a cascading effect. Such an outage should be resolved as quickly as possible, so early detection and alerting is key. In many cases the failover software used will be the most suitable to detect and alert on any outage, but in many cases the software is fairly limited in its functionality to alert. This generally means secondary monitoring systems are required to detect outages and alert on them. ClusterControl is not only able to detect outages on your database clusters, but can also pre-emptively detect (performance) issues that may arise. Apart from this it is also capable of sending out alerts.

Whether your team is using devops, chatops or traditional incident response management, you will need to get the communication flow going as quickly as possible. Many of us use external incident response applications like VictorOps or PagerDuty to get alerted and have (multiple) internal communication channels to collaboratively resolve these issues with your colleagues.  So how would you make sure your database alerts get propagated to each and every channel used? In this blog post, we will explain how to do this with the ClusterControl Integrations!

ClusterControl Integrations

Previously ClusterControl made it possible to propagate (critical) alerts and warnings to other systems, using our Notification Services framework. This required the user to install a nodejs service, which would send these messages to the user’s application. In some cases the nodejs service required some additional modifications.

In ClusterControl 1.4.2 we have overhauled this functionality and replaced it with ClusterControl Integrations. ClusterControl Integrations do not require any modifying of external files. In effect the integration with applications and services has now become so easy, that on average you can add a new integration in less than one minute!

To prevent you from receiving too many notifications, ClusterControl integrations also allow you to send out only specific critical alerts or warnings. This can be done in the same categorization as you are used to with our Email Notifications.

Current available integrations in ClusterControl are Pagerduty, VictorOps, Telegram, Opsgenie and Slack. If your application is not in the list, we also have a generic webhook that allows you to integrate ClusterControl into most existing applications.

For each of the integrations, we have a contextual help system that will guide you through the process.

As an example of how easy it is to do an integration, we will add two new integrations: Slack and VictorOps.

Adding the Slack integration

The Slack integration starts with going to the Apps & Integrations of your Slack group:

In this screen you can go to the Custom Integrations screen to add a new Incoming Webhook:

If you don’t see this app installed, you can search for it in the top Search App Directory field. Once selected you can install it for your Slack group and add the new Incoming Webhook from here.

You can have the webhook post to a user, a specific channel or just the #general channel in your Slack group. Once added, the Custom Integrations will show you the url you can post to:

Copy this URL and then paste it in the second ClusterControl Integrations step of Slack, and add a good descriptive name to the integration:

If you are uncertain if everything works, you can press the test button here to see if the webhook works.

In the third step you can select for which clusters and what type of messages you wish to send via the integration.

After we have added the integration, we can also verify in the edit screen if the integration is working. And as you can see, we have received a test message from ClusterControl in the designated channel.

Adding the VictorOps integration

Adding the VictorOps integration is very similar to the Slack integration: simply go to your VictorOps account and retrieve the REST integration url.

You can find the VictorOps REST integration by logging into VictorOps, go to Settings > Integrations. There will be a very long list of various integrations, but you must pick the REST integrations from here.

Then inside the VictorOps REST integration screen, you simply enable the integration. This will generate the URL that we need to use in ClusterControl.

After adding the URL to ClusterControl and naming the new integration, we can define which clusters and types of notifications we wish to send via the integration.

And just like with our Slack integration, we can verify if the integration works via the test button in the Edit screen. If everything goes well, you should see the test message from CusterControl appear on your timeline.

Conclusion

We have shown you how easy it is to integrate ClusterControl to your company communication channels, incident response- and workflow applications. We have added the most common applications, and provide a generic webhook to integrate others. Please let us know in the comments how this works for you, and in case you’d like to see integrations for other applications.