This is Part I of a multi-part Several9s blog series on how to use Severalnines’ ClusterControl REST API client SDK to programmatically control ClusterControl in order to accomplish many facets of database life cycle management and disaster recovery.

Currently, the client SDK has bindings for GoLang, Java and Python — we hope to release bindings for Ruby, Rust, Typescript, JavaScript, etc., in the near future. In this post, we will show how to use the auth, discovery and clusters endpoints. Let’s first look at the API.

The ClusterControl REST API overview

ClusterControl REST API documentation can be found at this link. It is possible to deploy and manage database clusters, take backups (and schedule and restore backups), etc., using the REST API. There are a number of endpoints to the REST API which we will cover in this post showing how to use them in a programmatic way in different programming languages.

N.B. The OpenAPI compliant specification of ClusterControl’s API is available in the clustercontrol_v2.yaml file in git.

N.B. We will focus on V2 (the most recent version at the time of writing) of the REST API.

The API server will be something like https://<clustercontrolhost>:9501/v2

There are a number of API endpoints servicing different aspects of ClusterControl functionality namely:

• v2/auth
• v2/jobs
• v2/backup
• v2/user
• v2/clusters
• v2/discovery
• v2/... 

To start, a user will have to authenticate with ClusterControl. via the v2/auth endpoint.

Authenticating with ClusterControl server

v2/auth : in order to use any endpoint, the API user will have to first authenticate and obtain a (CMON) SID (Session ID). The input to the request will be a json that looks like the following:

{
   "operation": "authenticateWithPassword",
   "user_name": "johnd",
   "password": "xxx"
}

The ClusterControl server will verify the user/password combination and, if successful, return a (CMON) SID in the cookie field of the HTTPS response along with the appropriate HTTP response code (e.g. 200). 

N.B. The SID will have to be sent to the ClusterControl server in the header field in subsequent calls to the rest of the API endpoints such as jobs, backup, user, etc.

Discovering supported database cluster types

v2/discovery : this endpoint can be used to discover the cluster types and their respective versions and distributions (e.g., Percona, MySQL, etc.) that ClusterControl supports. The input to the request will be a json that looks like the following:

{
   "operation": "getSupportedClusterTypes"
}

The discovery endpoint can also be used to verify if a cluster name is available. The input for such a request would be:

{
   "operation": "checkClusterName",
   "new_cluster_name": "test-cluster-001"
}

The response would have a field indicating whether the cluster name is already taken (or, not).

Obtaining information about deployed database clusters

v2/clusters : this endpoint can be used to obtain information about database clusters (all of the deployed clusters or a specific cluster based on the cluster name). The input to the request will be a json that looks like the following

Details of all clusters:

{
   "operation": "getAllClusterInfo",
   "with_hosts": true,
   "with_sheet_info": true
}

Details about a specific cluster:

{
   "operation": "getClusterInfo",
   "cluster_name": "test-cluster-001"
}

Now that we have shown the basic format of input for a few endpoints, let’s see how to configure the ClusterControl backend to service client REST calls.

Working with the ClusterControl API

The ClusterControl server needs to be configured in order for it to service client REST API calls. Perform the following on the ClusterControl host.

$ sudo vi /etc/default/cmon
# NEW

RPC_BIND_ADDRESSES="<INSERT IP here>,127.0.0.1"
# Insert the IP address of the CC host. It can be the hosts private or public IP address

$ sudo systemctl restart cmon

The git repository layout

The ClusterControl client SDK can be found in git. The client SDK repository has been laid out as follows:

./clustercontrol_v2.yaml : OpenAPI specification of ClusterControl API

./generated/go : GoLang bindings of the API

./generated/go/docs : Documentation of the GoLang bindings

Similarly for Java

./generated/java : Java bindings of the API

./generated/java/docs : Documentation of the Java bindings

And Python

./generated/python : Python bindings of the API

./generated/python/docs : Documentation of the Python bindings

The generated client binding needs to be adopted to work with ClusterControl’s authentication mechanism. The adopted code, and the code that you should use to work with ClusterControl are in the following respective directories for GoLang, Java and Python:

./go : GoLang bindings of the API adopted to work with ClusterControls’ authentication

./java : Java bindings of the API adopted to work with ClusterControls’ authentication

./python : Python bindings of the API adopted to work with ClusterControls’ authentication

Using GoLang bindings of the client SDK

The GoLang code can be easily built using the GoLang compiler. It also works with GoLand IDE. A sample application that shows how to authenticate with ClusterControl is available at ./go/cmd/clustercontrol-api-example 

./main.go : The main function for the sample go application.

package main
 
func main() {
  http.DefaultTransport.(*http.Transport).TLSClientConfig = &tls.Config{InsecureSkipVerify: true}
 
  opertest.AuthenticateWithCmon()
 
  opertest.Discovery()
}

./opertest/clustercontrol_v2.go : Contains the authenticate function which should be called first to authenticate with ClusterControl back-end.

package opertest
 
func AuthenticateWithCmon() {
  URL := os.Getenv("CC_URL")
 
  gCfg = NewConfiguration(URL)
  gApiClient = openapi.NewAPIClient(gCfg)
  authenticate := *openapi.NewAuthenticate("authenticateWithPassword")
  authenticate.SetUserName(os.Getenv("API_USER"))
  authenticate.SetPassword(os.Getenv("API_USER_PW"))
  ctx := context.Background()
  resp, err := gApiClient.AuthApi.AuthPost(ctx).Authenticate(authenticate).Execute()
  if err != nil {
     printError(err, resp)
     return
  }
  fmt.Fprintf(os.Stderr, "Resp `AuthApi.AuthPost`: %v\n", resp)
}

./opertest/discover_ops.go : Contains a sample discovery call to discover all of the supported cluster-types in ClusterControl.

package opertest
 
func Discovery() {
  discov2 := *openapi.NewDiscovery("getSupportedClusterTypes")
  resp, err = gApiClient.DiscoveryApi.DiscoveryPost(gNewCtx).Discovery(discov2).Execute()
  if err != nil {
     printError(err, resp)
     return
  }
  fmt.Fprintf(os.Stderr, "Resp `AuthApi.Discovery.GetSupportedClusterTypes`: %v\n", resp)
}

Using Java bindings of the client SDK

The Java code can be easily built using Maven using the pom.xml that is provided. It also works with IntelliJ IDE. A sample application that shows how to authenticate with ClusterControl is available ./java/clustercontrol-api-example

./src/main/java/com/severalnines/clustercontrol/clientsdk/cc/CcTest.java : Contains the authenticate method which should be called first to authenticate with ClusterControl back-end. Secondly, it also contains the discover method to discover all of the supported cluster-types in ClusterControl.

public void authenticateWithCmon() {
   String URL = System.getenv("CC_URL");
   String API_USER = System.getenv("API_USER");
   String API_USER_PW = System.getenv("API_USER_PW");
   System.out.println("CC_URL: " + URL);
   ApiClient defaultClient = Configuration.getDefaultApiClient();
   // defaultClient.setDebugging(true);
   defaultClient.setBasePath(URL);
   defaultClient.setVerifyingSsl(false);
 
   AuthApi authApiInstance = new AuthApi(defaultClient);
   Authenticate authenticate = new Authenticate(); // Authenticate | Authentication parameters
   authenticate.operation(Authenticate.OperationEnum.AUTHENTICATEWITHPASSWORD);
   authenticate.userName(API_USER);
   authenticate.password(API_USER_PW);
   try {
       // authApiInstance.authPost(authenticate);
       System.out.println("Auth request: " + authenticate.toString());
       ApiResponse<Void> resp = authApiInstance.authPostWithHttpInfo(authenticate);
       System.out.println("Auth response: " + resp.getData());
   } catch (ApiException e) {
       e.printStackTrace();
   }
}
 
private void discoveryTest() {
   ApiClient defaultClient = Configuration.getDefaultApiClient();
   DiscoveryApi discovApiInstance = new DiscoveryApi(defaultClient);
   Discovery discovery = new Discovery(); // Discovery | All things related to Clusters and cluster Hosts
   discovery.setOperation(Discovery.OperationEnum.GETSUPPORTEDCLUSTERTYPES);
   try {
       System.out.println("Discovery request: " + discovery.toString());
       ApiResponse<Void> resp = discovApiInstance.discoveryPostWithHttpInfo(discovery);
       System.out.println("Discovery response: " + resp.getData());
   } catch (ApiException e) {
       System.err.println("Exception when calling DiscoveryApi#discoveryPost");
       System.err.println("Status code: " + e.getCode());
       System.err.println("Reason: " + e.getResponseBody());
       System.err.println("Response headers: " + e.getResponseHeaders());
       e.printStackTrace();
   }
}

Using Python bindings of the client SDK

The Java code can be easily built using Python run time. It also works with Pycharm IDE. A sample application that shows how to authenticate with ClusterControl is available at ./python/clustercontrol-api-example

./python/clustercontrol_api_example/main.py :  Python main

from clustercontrol_api_example import authenticate
import requests
from requests.packages.urllib3.exceptions import InsecureRequestWarning
 
requests.packages.urllib3.disable_warnings(category=InsecureRequestWarning)
 
authenticate.authenticateWithCC()
authenticate.getClusterInfo()

./python/clustercontrol_api_example/authenticate.py :  The authenticateWithCC function which should be called first to authenticate with the ClusterControl back-end. The getClustersInfo function obtains information about a specific cluster (based on the cluster_name).

def authenticateWithCC():
# Enter a context with an instance of the API client
   print("authenticateWithCC():")
   with openapi_cc_client.ApiClient(configuration) as api_client:
       # Create an instance of the API class
       api_instance = auth_api.AuthApi(api_client)
       authenticate = Authenticate(
           operation="authenticateWithPassword",
           user_name = os.environ.get("API_USER"),
           password = os.environ.get("API_USER_PW"),
       ) # Authenticate | Authentication parameters
 
       # example passing only required values which don't have defaults set
       try:
           #kwargs = {"async_req": False}
           # Authenticate | Logout | Password Reset | Authenticate response (with challenge)
           resp = api_instance.auth_post(authenticate, async_req=False)
           print(resp)
       except openapi_cc_client.ApiException as e:
           print("Exception when calling AuthApi->auth_post: %s\n" % e)
 
def getClusterInfo():
   print("getClusterInfo():")
   with openapi_cc_client.ApiClient(configuration) as api_client:
       # Create an instance of the API class
       api_instance = clusters_api.ClustersApi(api_client)
       clusters = Clusters(
           operation="getclusterinfo",
           cluster_name="mysql-replication",
           with_hosts=True,
           with_sheet_info=True,
       )  # Clusters | Get cluster information
 
       # example passing only required values which don't have defaults set
       try:
           # GetClusterInfo | Get/Set Config | etc
           resp = api_instance.clusters_post(clusters, async_req=False)
           print(resp)
       except openapi_cc_client.ApiException as e:
           print("Exception when calling ClustersApi->clusters_post: %s\n" % e)

Wrapping up

Now you should be able to programmatically work with ClusterControl to authenticate and query for information about a specific cluster, and discover supported cluster types. We have shown the skeletons of the ClusterControl client SDK, which is derived from the OpenAPI specification of ClusterControl’s API, covering examples in GoLang, Java and Python.

As we’ve demonstrated, ClusterControl’s client SDK streamlines and simplifies working with ClusterControl. You can find the code in git, so make sure you kick the tires and provide feedback so that we can improve it. In the next part of this blog series, we will show how to deploy database clusters and take & restore backups.

Make sure you don’t miss out on the next post in the series and others by subscribing to our monthly newsletter below or following us on LinkedIn and Twitter.