Thursday 20 June 2013

Roadmap for Heat Havana (part 1)

It's been quite a while now since the design summit in Portland, and I've been meaning to write some details of the features we discussed at the summit, and in particular those which have appeared now on our plan for Heat's havana development cycle.

What follows are some highlights of what we're working on, or expect to be working on over the next weeks/months.  However I'll start with the disclaimer that this plan is a moving target, particularly since we're seeing an increasing number of new contributors whose planned contributions may not yet be captured on the plan, so please keep an eye on the plan in Launchpad for the latest details on what we're aiming to deliver - in other words, this may all change, but here-goes anyway! ;)

Concurrent Resource Scheduling

Inside heat, we create a dependency graph of resources, which we use to determine ordering of all operations (for example create, update, delete) within stack.  For grizzly, the order in which these actions happen is determined via topological sorting followed by performing each action in series.

Clearly this is far from ideal when you have large numbers of non-dependent operations (for example creating a large number of grouped instances), so work has been under-way to improve this situation and perform stack operations in parallel where possible.  The initial focus of this work is resource creation, but the plan is to eventually perform as many stack operations as possible concurrently, making use of the new task scheduler that has been developed, which uses coroutine based task scheduling.

Related to this work, we've been discussing ideas with the wider community around requirements for workflow and task scheduling in Heat, so that hopefully we can figure out a way to use a common solution across projects with these sorts of requirements.

Stack Suspend/Resume

This feature is aimed at allowing coordinated suspend/resume of a group of resources (including nested resources), such that you can take either an entire stack, or individual resources offline and the resume them (quickly) at some later time.

The idea is to provide access to the some of the underlying capabilities provided by the nova admin actions API, but to use the dependency information we have in Heat to do things in the correct order wrt the stack definition.  

We will also handle non-instance suspend operations, for example disabling defined alarms, such that suspend/resume can be performed in a non-destructive way.

We may also provide access to other actions in future, so considerable effort has gone into refactoring such that this should be possible with less effort and duplication.

Native Template Language (Heat Orchestration Template aka "HOT")

So, this ended up being *the* hot topic at the design summit in Portland, we got the message, loud and clear, that there are a lot of users, and potential users, who would like to see an openstack-native (non CFN-compatible) template language develop.

There are two threads to this work - firstly defining the missing logical abstractions (ie what cannot be adequately expressed via the current heat logical model), and secondly the syntax itself.  Most of these efforts are captured as dependencies of this umbrella blueprint, and there is the syntax specific "HOT hello world" effort.

This is a large and complex piece of work, and the progress made so far has been good, in particular there are recently aspects of the Environments and Providers abstractions landing, which will enable the future work to progress.  (I'll post again next week with further details on these aspects)

To be continued...

That's all I have time for today, but hopefully provides a taste of what we've got in the pipeline for havana - there's more, much more (ceilometer integration, keystone trusts, native resource types, engine scale-out, concurrent updates, rolling updates, etc etc!), but I'll have to cover those another time! :)