Heat auth model updates - part 1 Trusts

Over the last few months I've spent a lot of my time looking at ways to rework the heat auth model, in an attempt to solve two long-standing issues:

1. Requirement to pass a password when creating a stack which may perform deferred orchestration actions (for example AutoScaling adjustments)
2. Requirement for users to have administrative roles when creating certain types of resource.

So, fixes to these issues have been happening (in Havana and Icehouse respectively), but discussions with various folks indicates significant confusion re differentiating the two changes, probably because I've not got around to writing up the documentation yet (it's in progress, honest!) ;)

In an attempt to clear up the confusion, and provide some documentation ahead of the upcoming Icehouse Heat release, I'm planning to cover each feature in this and a subsequent post - below is a discussion of the "Requirement to pass a password" problem, and the method used to solve it.

What? Passwords? Don't we pass tokens?

Well, yes mostly we do.  However the problem with tokens is they expire, and we have no way of knowing how long a stack may exist for, so we can't store user tokens to do deferred operations after the initial creation of the heat stack (not that it's a good idea from a security perspective either..)

So in previous versions of heat, we've required the user to pass a password (yes, even if they are passing us a token), which we'd then encrypt and store in the heat database, such that we can then obtain a token to act on behalf of the user and to whatever deferred operations are required during the lifetime of the stack.  It's not a nice design, but when it was implemented, Trusts did not exist in Keystone so there was no viable alternative.  Here's exactly what happens:

• User requests stack creation, providing a token and username/password (python-heatclient or Horizon normally requests the token for you)
• If the stack contains any resources marked as requiring deferred operations heat will fail validation checks if no username/password is provided
• The username/password are encrypted and stored in the heat DB
• Stack creation is completed
• At some later stage we retrieve the credentials and request another token on behalf of the user, the token is not limited in scope and provides access to all roles of the stack owner.

Clearly this is suboptimal, and is the reason for this strange additional password box in horizon:

You already entered your password, right?!

Happily, after discussions with Adam Young, Trusts were implemented during Grizzly and Heat integrated with the functionality during the Havana cycle.  I get the impression not that many people have yet adopted it, so I'm hoping we can move towards making the new trusts based method the default, which has already happened for devstack quite recently.

Keystone Trusts 101

So, in describing the solution to Heat storing passwords, I will be referring to Keystone Trusts, because that is the method used to implement the solution.  There's quite a bit of good information out there, including the Keystone Wiki, Adam Young's blog and the API documentation, but here's a quick summary of terminology which should be sufficient to understand how we're using trusts in Heat:

Trusts are a keystone extension, which provide a method to enable delegation, and optionally impersonation via keystone.  The key terminology is trustor (the user delegating) and trustee (the user being delegated to).

To create a trust, the trustor (in this case the user creating the heat stack) provides keystone with the following information:

• The ID of the trustee (who you want to delegate to, in this case the heat service user)
• The roles to be delegated (configurable via the heat configuration file, but it needs to contain whatever roles are required to perform the deferred operations on the users behalf, e.g launching a nova instance in response to an AutoScaling event)
• Whether to enable impersonation

Keystone then provides a trust_id, which can be consumed by the trustee (and only the trustee) to obtain a trust scoped token.  This token is limited in scope such that the trustee has limited access to those roles delegated, along with effective impersonation of the trustor user, if it was selected when creating the trust.

Phew! Ok so how did you fix it?

Basically we now do the following:

• User creates a stack via an API request (only the token is required)
• Heat uses the token to create a trust between the stack owner (trustor) and the heat service user (trustee), delegating a special role (or roles) as defined in the trusts_delegated_roles list in the heat configuration file.  By default heat sets this to "heat_stack_owner", so this role must exist and the user creating the stack must have this role assigned in the project they are creating a stack.  Deployers may modify this list to reflect local RBAC policy, e.g to ensure the heat process can only access those services expected while impersonating a stack owner.
• Heat stores the trust id in the heat DB (still encrypted, although in theory it doesn't need to be since it's useless to anyone other than the trustee, e.g the heat service user)
• When a deferred operation is required, Heat retrieves the trust id, and requests a trust scoped token which enables the service user to impersonate the stack owner for the duration of the deferred operation, e.g to launch some nova instances on behalf of the stack owner in response to an AutoScaling event.

The advantages of this approach are hopefully clear, but to clarify:

• It's better for users, we no longer require a password and can provide full functionality when provided with just a token (like all other OpenStack services... and we can kill the Horizon password box, yay!)
• It's more secure, as we no longer store any credentials or other data which could use used by any attacker - the trust_id can only be consumed by the trustee (the heat service user).
• It provides much more granular control of what can be done by heat in deferred operations, e.g if the stack owner has administrative roles, there's no need to delegate them to Heat, just the subset required.

I'd encourage everyone to switch to using this feature, enabling it is simple, first update your heat.conf file to have the following lines:

deferred_auth_method=trusts

trusts_delegated_roles=heat_stack_owner

Hopefully this will soon become the default from Juno for Heat.

Then ensure all users creating heat stacks have the "heat_stack_owner" role (or whatever roles you want them to delegate to the heat service user based on your local RBAC policies).

That is all, more coming soon on "stack domain users" which is new for Icehouse and resolves the second problem mentioned at the start of this post! :)