Windows and Heat

As OpenStack is gaining traction in the enterprise world, the number of users asking for Microsoft Windows instances support is progressively increasing with a fast pace.

Most of the tools involved traditionally in OpenStack guest provisioning and orchestration are Linux specific and need to be ported, integrated or replaced with other alternatives in order to support Windows guests.

Furthermore, most typical Windows workloads require deployment automation mechanisms that differ from their Linux counterparts, needing reboots, access to installation media (e.g. ISO images), passwords, licensing keys and more as detailed in the following paragraphs.

Cloudbase-Init can be considered as the de facto standard OpenStack Windows guest initialisation tool, providing features similar to cloud-init. The project started as a complete separate implementation, since a port of cloud-init has been proven unfeasible due to the platform differences and the tight coupling with the Linux architecture.

Cloudbase-Init is platform independent and beside the stock Windows support, a FreeBSD fork has been recently published by third parties.

A Windows installer is available to setup the application in a few simple steps, including Sysprep integration.

Support for metadata currently includes Nova (HTTP), ConfigDriveV2 and EC2. Features are provided in the form of an extendible plugin framework, currently including:

• Host name management
• User creation / groups management
• User password generation
• Network configuration
• SSH public keys deployment
• Volumes automatic extension
• WinRM server configuration
• WinRM password-less certificate authentication
• User data scripts (including multi-part support)

User data management includes support for executing PowerShell, Command line batches (CMD/BAT) and Bash scripts, along with multipart support, which in turn includes Heat support.

Different content types available in multipart user data workloads are handled via an extensible plugin framework, which currently includes text/x-shellscript, text/part-handler and text/x-cfninitdata content.

On Linux, Heat support is currently managed via a part-handler and a shellscript for Heat user data execution, not supporting Windows at the present stage. To overcome this limitation for the time being, Heat data is managed by the text/x-cfninitdata plugin, providing the same features supported by the part-handler counterpart.

Heat support can thus be considered complete in Cloudbase-Init.

Here's an example of Heat user data, showing Linux specific Python code: http://paste.openstack.org/show/62614/

All the operations executed by Cloudbase-Init are logged to file and can be optionally logged to a serial port. The latter allows to perform troubleshooting by accessing the console log in Horizon or via nova console-log.

Windows workloads requirements differ in some circumstances from similar Linux cases. Here's a list of the most typical scenarios.

Unfortunately actions like changing the host name or joining an Active Directory domain still require a reboot on Windows.

In order to integrate the reboots seamlessly in the Heat provisioning workflow, the user data script needs to signal to Cloudbase-Init the intention to do so, specifying also if the script should be executed or not during the next boot.

This is accomplished by exiting the script with a special value:

• 1001: Reboot and don't execute the user data plugin during the next boot
• 1003: Reboot and execute the user data plugin during the next boot

The user data script can keep track of the current state before a reboot by using custom registry values in order to resume execution at the desired point during the next boot.

Access to Linux instances can be performed via SSH without using a password by employing public key authentication and a keypair deployed via Nova. Windows can use client certificate authentication with WinRM for this objective, as described here. This is already supported by Cloudbase-Init and a Nova blueprint for handling certificates in a way similar to keypairs is under development and can be easily supported by Heat. As an alternative, the certificate could be passed in the Nova metadata multipart user_data for compatibility with the existing OpenStack deployments.

Windows user passwords can be generated during the execution of Cloudbase-Init, encrypted with the SSH keypair's public key and retrieved with nova get-password, thus avoiding any clear text transmission in the process.

The same does not apply for example in the case of domain credentials required to join a virtual machine resource to an Active Directory domain.

A partial solution for avoiding the need to specify clear text passwords as template parameters is to deploy a keypair in the instance, use the public key to encrypt the sensitive data and subsequently the private key during user data script execution for decryption. The main security limit of this option is that all instances of a given image will share the same keypair.

New volumes are attached unpartitioned, so they are not directly visible from applications or command line. A new volume must be set online, initialized and partitioned. New partitions need also to be formatted and if necessary a drive letter can be assigned as well.

The following example performs all the required activities, including assigning the “E” drive letter to the new partition, labelled “Volume1”. In case an automatic drive letter assignment should be preferred, just replace -DriveLetter <letter> with -AssignDriveLetter.

$d = Get-Disk | where {$_.OperationalStatus -eq "Offline" -and $_.PartitionStyle -eq 'raw'}
$d | Set-Disk -IsOffline $false
$d | Initialize-Disk -PartitionStyle MBR
$p = $d | New-Partition -UseMaximumSize -DriveLetter "E"
$p | Format-Volume -FileSystem NTFS -NewFileSystemLabel "Volume1" -Confirm:$false

Most Microsoft workloads deployments require access to install media (ISO images), for Example Microsoft Exchange Server, SharePoint or SQL Server, which cannot be simply downloaded on the instance due to size and licensing constraints. A solution consists in placing the images on a volume to be attached to the virtual machine resources defined in the template.

Here's a simple CFN template snippet showing how to use PowerShell to configure the first Active Directory domain controller in a forest and requesting a reboot once done:

"UserData"       : { "Fn::Base64" : { "Fn::Join" : ["", [
  "#ps1_sysnative\n",
  "$ErrorActionPreference = 'Stop'\n",
  "Install-WindowsFeature -Name AD-Domain-Services -IncludeManagementTools\n",
  "$user = [ADSI]'WinNT://./Administrator'\n",
  "# Disable user\n",
  "#$user.userflags = 2\n",
  "#$user.SetInfo()\n",
  "$user.SetPassword('", { "Ref" : "SafeModeAdministratorPassword" }, "')\n",
  "Import-Module ADDSDeployment\n",
  "$safeModePwd = (ConvertTo-SecureString '", { "Ref" : "SafeModeAdministratorPassword" }, "' -AsPlainText -Force)\n",
  "Install-ADDSForest -DomainName '", { "Ref" : "DomainName" }, "' -DomainNetbiosName '", { "Ref" : "DomainNetbiosName" }, "' -SafeModeAdministratorPassword $safeModePwd -InstallDns -NoRebootOnCompletion -Force\n",
  "exit 1001\n",
]]}}

More Windows Heat templates examples are available here: https://github.com/cloudbase/windows-heat-templates

An OpenStack implementation of the AWS CloudFormation bootstrapping tools is available: https://github.com/openstack/heat-cfntools

The heat-cfntools are currently not supported on Windows. Implementing the platform specific features is not a particularly complicated task and would require as a minimum:

• Platform specific configuration path (“C:\cfn\”)
• Packaging MSI support
• Windows services configuration
• Windows commands

As an alternative, the original AWS CloudFormation Apache 2 licensed sources can be used as a starting point: http://aws.amazon.com/developertools/AWS-CloudFormation/4026240853893296