TripleO Heat templates Part 3 - Cluster configuration, introduction/primer

In my previous two posts I covered an overview of TripleO template roles and groups, and specifics of how initial deployment of a node happens.  Today I'm planning to introduce the next step of the deployment process - taking the deployed groups of nodes, and configuring them to work together as clusters running the various OpenStack services encapsulated by each role.

This post will provide an introduction to the patterns and Heat features used to configure the groups of nodes, then in the next instalment I'll dig into the specifics of exactly what configuration takes place in the TripleO heat templates.

Recap - the deployed group of servers

So, we're continuing from where we got to at the end of the last post - we've deployed a ResourceGroup containing several OS::TripleO::Controller resources, which in turn have deployed a nova server, and done some initial configuration of it.

What comes next is configuring the whole group, or cluster, to work together, e.g configuring the OpenStack services running on the controller.

Group/Cluster configuration with Heat

Similar to the SoftwareDeployment (singular) resources described in my previous post, Heat supports applying a SoftwareConfig to a group of servers via the SoftwareDeployments and StructuredDeployments (plural) resources.  The function of both is basically the same, one works with a SoftwareConfig resource and the other with a StructuredConfig resource.

Typically (in TripleO at least) StructuredDeployments resources are used combined with a ResourceGroup containing some servers.  You pass a list of servers to configure (provided via an attribute from the OS::Heat::ResourceGroup resource), and a reference to a StructuredConfig resource.

The StructuredConfig resource defines the configuration to apply to each server, and the StructuredDeployments resource then internally creates a series of StructuredDeployment (singular) resources, one per server.

When all of the deployment (singular) resources complete, the deployments (plural) resource goes CREATE_COMPLETE - if any of the nested deployment resources fail, the deployments resource will go into a FAILED state.

Debugging groups of deployments

You may notice that the StructuredDeployments resource above looks a lot like the ResourceGroup containing the OS::TripleO::Controller resources - this is no coincidence, internally heat actually creates a ResourceGroup containing the StructuredDeployment resources.

This is a useful fact to remember when debugging, because it means you can use the techniques I've previously described to inspect the individual Deployment resources  created by the StructuredDeployments resource, e.g so you can use heat deployment-show <id> to help diagnose a problem with a failing deployment inside the StructuredDeployments group (which is often quicker and more convenient than SSHing onto the failing node and trawling the logs).


For example, here's a simple bash script which dumps out details about all of the Deployment resources in an overcloud, obviously you can add in a "grep FAILED" here if you just want to see details about failing deployments:

#!/bin/bash
while read -r line
do
  deployment_name=$(echo $line | cut -d"|" -f2)
  deployment_id=$(echo $line | cut -d"|" -f3)
  parent_name=$(echo $line | cut -d"|" -f7)
  echo "deployment=$deployment_name ($deployment_id) parent $parent_name"
  heat deployment-show $deployment_id
  echo "---"
done < <(heat resource-list --nested-depth 5 overcloud | grep "OS::Heat::\(Software\|Structured\)Deployment ")

We should probably add a python-heatclient feature which automates this lookup (particularly for failing deployments), but right now that is one way to do it.

 

Until next time..!

So here we've covered the basics of how Heat can be used to configure groups of servers, and we've illustrated how that pattern is applied in the TripleO templates.

The TripleO templates use this technique for all roles, to do multiple configuration passes during the deployment - in the next post I'll cover specifics of how this works in detail, but for now you can check out the TripleO heat templates and hopefully see this pattern for yourself.  Note that it's combined with provider resource abstractions as previously discussed, which as we will see makes for a nicely abstracted approach to cluster configuration which is pretty easy to modify, extend, or plug in alternative implementations.