Automating New Servers

Provisioning a new server used to be a long, expensive and drawn out process, including shipping new hardware, installing it in the datacenter and cabling before even powering the box on to configure the software element.  This took a step forwards with the adoption of using virtual machines to mitigate the new hardware aspect, but still required specialist knowledge to configure the VM (especially if it was an on-premises server farm).  The explosion in Cloud Computing, with offerings from Amazon, Microsoft, Rackspace and others, means that it is possible to get up and running with a new Virtual Machine in a matter of minutes, with very little technical expertise required.  Investing relatively little time up front can automate the remaining configuration and provide an interface that means ownership of an entire process can be delegated to teams of varying skill sets.  This also provides the autonomy required for an Agile process to succeed.

In this post I’m going to walk through some specific aspects of Microsoft’s Azure platform and how they can be utilized to spin up a new server with all the dependencies to run a Sitecore site.

The Azure Portal is a great UI for managing individual resources, however when starting to build out a new environment, or make change in bulk, it is not the best tool.  This is where Azure Powershell comes into its own.  There are many ways to integrate with Azure APIs – they are just REST endpoints after all, and if you’re more comfortable writing C#, there are SDKs on NuGet, but powershell lends itself to a DevOps culture as it spans the programming knowledge of a Development team, while also incorporating the knowledge that is embedded in Ops team.

The Azure Resource Manager is the ‘new’ way of creating and managing resources created in the Azure Cloud.  ARM templates are a way of defining a single resource, or an interlinked number of resources together in a JSON format.  This allows you to create anything from a single public IP up to an entire environment, with multiple servers, load balancers, SQL PaaS instances and so on.   The Azure Portal will even generate a file based on existing resources that can then be parameterized to be more generic.

An ARM template is made up, mostly, of 3 properties that take an array of objects;

  • Parameters – values passed into the template to customize it for the resource being deployed.
  • Variables – often made up of concatenations of parameters or the result of functions, variables simplify the overall structure of the template.
  • Resources – the definition of the resources to be created. These can reference variables and parameters, as well as other resources.

When defining a Virtual Machine, it is important to consider the cascading dependencies that are required – at the most basic level, you’ll need a storage device for the OS and a network interface.  The network interface will need public and private IP addresses as well as a Network Security Group (essentially a representation of firewall rules) and subnet.  This could also be made to be more complex to include Availability Sets, multiple disks and NICs etc.

Many items will be unique and distinct for a new VM – the Disks, IPs and NICs for example would never be shared between VMs.  A Network Security Group or a subnet, however, could well be shared across a number of machines.  At this point it is wise to decide if you will manage your infrastructure fully through ARM templates, through the portal or as a hybrid approach.  The reason this decision is important is because when deploying a resource via the ARM templates, if a dependent resource already exists in the Azure environment, it will be modified to match the template.  If you consider a Network Security Group that is defined in an ARM template, if someone goes and modifies that Group via the portal, and they do not update the template, the next time the template is used to create a resource, the change will be reverted.

A simple workaround for this is to not define all dependent resources in the ARM template, but to reference some things by ID. While the template is JSON formatted, it is possible to reference standard functions that are understood by the ARM infrastructure.  One of these is the resourceId() function, which accepts two parameters – the type of resources, and the name.  This will return the Azure ID for that resource, which can be used as a reference for new resources.

This simplifies the creation of a single server, and gives massive efficiency improvements when creating multiple servers.  There are also a large number of extensions that can be applied that automate many post-initialization tasks.  One of the most helpful out of the box is the JsonADDomainExtension which, as the name implies, connects the server to a Domain by providing a Domain Admin account – these can, and should, be parameters passed into the script.

It is also possible to enable custom extensions which can run a file stored in blob storage – this could be another Powershell script, an executable or a batch file.  The script/executable is downloaded and launched directly on the server, so it is possible to write scripts as if you were running them locally. We use this to add an AD group to the local Admin group on the server, install Windows features, set the time zone, and install common tools via chocolatey.

Once you have your json template defined, creating the new resources is a trivial task;

$resourceGroupName = "<your resourc egroup>"
$templateFilePath = "<Path to your json template>
$params = @{
     storageAccountName="<your account name>";
     adminPassword="a$ecurepa$$word";
     adminusername="<admin username>";
     vmName="<VMName>";
     vmSize="<VMSize>"
}
New-AzureRmResourceGroupDeployment -ResourceGroupName $resourceGroupName -TemplateFile $templateFilePath -TemplateParameterObject $params -Mode Incremental -Name $vmName;

These values can be, generally, anything you want them to be.  The exception to this is the vm size, which must match a value from the currently offered list of VMs, found here