Saturday, May 16, 2015

I am speaking at TechDays 2015

Happy to announce that I am speaking at Techdays 2015 Netherlands on automating the Azure infrastructure. Here are the details of my session

Managing your Azure infrastructure with Chef and DSC

Date and time: 29-May-2015, 15:15 - 16:15
Venue: North America

I'll be showing how to leverage the capabilities of Chef and DSC to automate windows Azure infrastructure. DSC is a way to automate infrastructure management using a configuration driven approach where the desired state of the machine/ machines can be configured and ensure that the machines adhere to it all the times. Chef provides a cross-platform ecosystem of tools and patterns for taking DSC from feature to full solution. The combination of Chef and DSC make a powerful companion for managing your Azure infrastructure. In this session, we'll see how the combination enables you to work effectively with your cloud infrastructure. Can’t wait to see you all there!

Managing your Azure infrastructure as code - part 4 (Chef and DSC)

Chef and DSC

DSC is a new management platform in Windows PowerShell that enables deploying and managing configuration data for software services and managing the environment in which these services run.
DSC provides a set of Windows PowerShell language extensions, new Windows PowerShell cmdlets, and resources that you can use to declaratively specify how you want your software environment to be configured. It also provides a means to maintain and manage existing configurations.
With the release of Chef Client 11.16.0 PowerShell Desired State Configuration (DSC) support was introduced into Chef Client for Windows. With this feature DSC resources can be executed from Chef. Both DSC and chef are similar in the way that both are idempotent, take similar approaches to the concept of resources, describe the configuration of a system, and then take the steps required to do that configuration. The most important difference between Chef and DSC is that Chef uses Ruby and DSC is exposed as configuration data from within Windows PowerShell. The dsc_resource resource allows any DSC resource to be used in a Chef recipe, as well as any custom resources that have been added to your Windows PowerShell environment.
The syntax for using the dsc_resource resource in a recipe is as follows:
dsc_resource "name" do
  resource :dsc_resource_name
  property :dsc_property_name, "property_value"
  property ...
  ...
end

For e.g the below given code shows how the windowsfeature resource is used to install asp.net 4.5 on the node.
dsc_resource 'aspnet' do
   resource :windowsfeature
   property :ensure, 'Present'
   property :name, "web-asp-net45"
 end

LCM and WMF 5.0

Before using the dsc_resource, you need to ensure that Windows Management Framework 5.0 is installed on the target node. Also the refresh mode of the LCM should be set to disabled. To set the LCM refresh mode to disabled, you can create a configuration as given below and execute it.
Configuration LCMSettings {
    Node $env:COMPUTERNAME {
       LocalConfigurationManager {
           RefreshMode = "Disabled"
       }
    }
}

LCMSettings



Creating a cookbook for using DSC resources

To use the DSC resources, create a cookbook and edit the default.rb file in recipes folder to add the content as given below.

dsc_resource 'iis' do
   resource :windowsfeature
   property :ensure, 'Present'
   property :name, "web-server"
 end
 dsc_resource 'aspnet' do
   resource :windowsfeature
   property :ensure, 'Present'
   property :name, "web-asp-net45"
 end
 dsc_resource 'defaulthtml' do
                resource :file
                property :ensure, 'Present'
                property :type, 'File'
                property :destinationpath, 'C:\inetpub\wwwroot\index.htm'
                property :contents, 'html of the page'
 end
dsc_resource 'iisstart' do
                resource :file
                property :ensure, 'Absent'
                property :type, 'File'
                property :sourcepath, 'C:\inetpub\wwwroot\iisstart.htm'
                property :destinationpath, 'C:\inetpub\wwwroot\iisstart.htm'
 end

Upload the cookbook to the server and chef the chef server whether the cookbook is available.

Bootstrapping the chef client

To apply the configuration on the machine, use the command
knife bootstrap windows winrm cheftestvm00.cloudapp.net -x ‘user’ -P 'password' -r "recipe[azurewebrole]"
After executing the command, you can access the default url for the server and check whether the contents match the one given in the file resource.




Managing your Azure infrastructure as code - part 3 [Chef cookbooks]

Cookbooks

In Chef cookbooks serve as the fundamental unit of configuration and policy distribution. Chef uses cookbooks to perform work and make sure things are as they should be on the node. Cookbooks are the way Chef users package, distribute, and share configuration details. A cookbook usually configures only one service. For e.g. the general structure of a cookbook contains of the elements as given below (attributes, definition, files, recipes etc.)

The important concepts for a cookbook are attributes and recipes. Attributes are defined in the cookbook that can be used to override the default settings on a node. When a cookbook is loaded during a chef-client run, these attributes are compared to the attributes that are already present on the node. Attributes that are defined in attribute files are first loaded according to cookbook order. For each cookbook, attributes in the default.rb file are loaded first, and then additional attribute files (if present) are loaded in lexical sort order. When the cookbook attributes take precedence over the default attributes, the chef-client will apply those new settings and values during the chef-client run on the node. Recipes are Ruby files in which you use Chef's Domain Specific Language (DSL) to define how particular parts of a node should be configured. The default.rb file will be run through first to install the service, followed by module recipes. Inside the recipe the content of the default.rb file looks like:
powershell_script 'Install IIS' do
    action :run
    code 'add-windowsfeature Web-Server'
end

service 'w3svc' do
    action [ :enable, :start ]
end

The chef-client will run a recipe only when asked. When the chef-client runs the same recipe more than once, the results will be the same system state each time. When a recipe is run against a system, but nothing has changed on either the system or in the recipe, the chef-client won’t change anything.

Creating and uploading a cookbook

To create a cookbook you can use the generate cookbook command. Login to the chef workstation and use the PowerShell console to navigate to the C:\Chef directory that was created earlier and run the command.
chef generate cookbook [[cookbookname]]

The command will create the cookbook with the files and folder structure for the components for cookbook. These files are generated under the cookbook directory in the Chef directory of the workstation. In our example this is the C:\chef\cookbooks folder.
Next you need to add the commands that will be executed when the cookbook is executed. These commands are stored in the default.rb file in the recipes folder. To add these commands edit the default.rb file in the ~\cookbooks\recipes folder and add the content like
file 'hello.txt' do
  content 'Welcome to Chef'
end
The cookbook when executed will create a file hello.txt with the content “Welcome to Chef”. To upload the cookbook to the chef server you can use the knife cookbook upload [[cookbookname]] command.


Bootstrapping a windows server

A bootstrap is a process that installs the chef-client on a target system so that it can run as a chef-client and communicate with a Chef server.
Use the knife bootstrap subcommand to run a bootstrap operation that installs the chef-client on the target system and also apply the cookbook on the client machine.

knife bootstrap windows winrm ‘servername’ -x ‘adminuser’ -P 'password' -r "recipe[cookbookname]"

Sunday, May 3, 2015

Managing your Azure infrastructure as code - Part 2 (Chef)

In the previous post on managing azure infrastructure in code, we've seen how to leverage the capabilities of PowerShell to fully automate provisioning your Azure infrastructure. By using the capabilities of PowerShell its now easy to define the set of polices in code or a DSC configuration file, that can be used to provision the infrastructure and bring it to the desired state. With this, the effort required to manually install and configure components or software on the machines are eliminated and can be done in a consistent manner across the DTAP street.

With the introduction of tools like Chef, Puppet etc. the focus on automation has been improved and can give a completely new experience in managing the infrastructure. In this post, we'll see how to the combination of Chef and Azure can be utilized to deliver infrastructure as code that can be used to manage the Azure VM's.

About Chef:

Before we dive into the details, we'll have a quick overview of Chef and the associated components.
The main components of a Chef infrastructure is Server, Workstation, client and nodes as given below.

  1. The Chef server acts as a hub for configuration data. The Chef server stores cookbooks, the policies that are applied to nodes, and metadata that describes each registered node that is being managed by the chef-client. Nodes use the chef-client to ask the Chef server for configuration details, such as recipes, templates, and file distributions. Clients communicate with the server to get the right configuration elements from the server and apply it to the nodes.
  2. The workstation is the development machine from which configuration elements like cookbooks, recipes and policies are defined. Configuration elements are synchronized with the chef-repo and uploaded to the server with knife command. The knife commands are executed from the Chef Workstation to manage our infrastructure.
  3. A node is any physical, virtual, or cloud machine that is configured to be maintained by a chef-client. The Chef client is an agent that sits on the servers that needs to be managed.

To start with the Chef infrastructure, you need to first manage the Chef server. There are 3 types of Chef servers.
  1. Hosted Chef: The is the server that is hosted by Chef. For most of scenarios you should be able to use this server. For our example we’ll be using the hosted server.
  2. Enterprise Chef: This server is available on premise and has to be managed by the you.
  3. Open Source Chef: is a free version of Chef Server.


Managing your Chef Infrastructure:

You can sign up free for the hosted chef server at https://manage.chef.io/signup. After the signup process, you can download the starter kit that can be installed at the workstations to manage your cookbooks and other resources. You need to copy the files from the starter kit to a folder in your workstation, that will used to manage the configuration, keys etc. for the chef infrastructure. 



To manage the Azure infrastructure, Chef needs to access the publish settings file from your Azure subscription. You can run the Get-AzurePublishSettingsFile cmdlet at the PowerShell prompt in your Azure machine to download the settings file. The file should be copied to the workstation and added the knife.rb file to specify the chef specific configuration details for the knife. Refer to this link at Chef documentation for more details on the knife.rb file https://docs.chef.io/config_rb_knife.html.

To manage the Azure infrastructure we’ll use the azure-knife resource, a knife plugin to create, delete, and enumerate Microsoft Azure resources to be managed by Chef. This plugin is distributed as a Ruby Gem. To install it, run: gem install knife-azure at the command prompt. Depending on your system's configuration, you may need to run this command with root/administrator privileges. For this plugin to interact with Azure's REST API, you will need to give Knife information about your Azure account and credentials as knife[:azure_publish_settings_file] = ".publishsettings". The knife.rb file in my workstation looks like.


Creating your Azure Infrastructure:
The knife azure create server command can be used to create an Azure VM using chef.
You can create a server with minimal configuration. On the Azure Management Portal, this corresponds to a "Quick Create - VM". For e.g this command will create a Azure VM with the details given below.

knife azure server create
            --azure-dns-name 'cheftest01'
            --azure-service-location 'West Europe'
            --azure-source-image 'source-image-name-of-windows-server'
            --winrm-user adminusername
            --winrm-password 'adminpassword'

You can use the Get-AzureVMImage |? {$_.Label -like 'Windows Server 2012 R2 Datacenter*' }| select imagename cmdlet to see the Azure images available.

You can also set various other options in the advanced create. Eg: If you want to set the Azure VM Name different from that of the Azure DNS Name, set the option :azure_vm_name. If you want to specify the subscription id etc.

knife azure server create
          --azure-subscription-id ''
          --azure-dns-name 'ChefTestVM01'
          --azure-vm-name 'ChefTestVM01'
          --azure-vm-size 'Small'
          --bootstrap-protocol 'cloud-api'
          --azure-source-image 'a699494373c04fc0bc8f2bb1389d6106__Windows-Server-2012-R2-201504.01-en.us-127GB.vhd'
          --azure-service-location 'West Europe'
          --winrm-user
          --winrm-password
          --tcp-endpoints 80,3390

Once you run the command, you can see the details either at the command prompt or at the management portal for Windows Azure.


Once the deployment is complete, we should be able to see the virtual machine in your available VM list and should be able to connect to it and use it as any other VM's.