Friday, July 31, 2015

Setup your Chocolatey repository in Azure - Part 1


The complete list of posts in this series:

Part 1: Creating a nuget website to upload packages
Part 2: Packaging and uploading packages to the nuget repository
Part 3: Configuring PowerShell package management or OneGet to install packages from your repository
Part 4: Configure the infrastructure in Chef
Part 5: Using DSC and Chef to install the packages

Chocolatey is a package manager for Windows designed to be a decentralized framework for quickly installing applications and tools that you need. Chocolatey is actually built on top of the NuGet package system, but it is designed to fill a different need. Chocolatey wraps up applications and other executables and makes it easy to install them on your computer.

The goal behind this post is to successfully setup up a Chocolatey repository in Azure websites to distribute software easily throughout the network. You can use the same approach to host a repository on premises, by choosing a deployment option not to host on Azure but on local IIS.

Follow the steps in the given order to complete the process.

  • Open Visual Studio and create an empty project (ASP.NET Web Application)

  • In the package manager console, type the cmdlet Install-Package Nuget.Server to install the NuGet server package

  • Alternatively you can use the Manage Nuget Packages options by right clicking the project.

  • After successful installation, your project structure should look like

  • Open the added Web.config file and change the following values in the appSettings section. Make sure that your package path is not a physical path like C:\MyPackages. With Azure websites, you will not have write access to the c:\ on the machines.

  • Right click the project and choose Publish website to deploy the website to Azure

  • In the dialog, sign into your Azure subscription and provide values for the web site name as given below

  • Continue the Wizard and click on Publish to start the deployment process.


  • After successful deployment you can browse the website from the url given for the website.

  • Now we have the source repository configured, next we can use this source to publish and install packages we need. That we’ll see in the next post.


Wednesday, July 29, 2015

Build and publish to NuGet as part of TFS build process

NuGet is a free, open source package management system for .NET platform, which helps in easy dependency management. Using NuGet very easy for project owners to build and define package dependencies which will be included in the package during the packaging process and later publish this to the repository for usage by external users.

Users can later install the package by searching in the NuGet source repository which will automatically download and install the dependencies as well. You can use the NuGet command line utility to create packages from a specification file and later publish them to the repository whenever needed. If you want to automate the process of package creation and publishing from a TFS build process, you can make use of custom PowerShell scripts that can be invoked as part of the build process. The Team Foundation Build template (TfvcTemplate.12.xaml) provides the capability to extend the build process by using PowerShell scripts at stages like Pre build and Post build.

You can use the default template to run PowerShell and batch (.bat) scripts before and after you compile your code, and before and after your run your tests. During the script execution you can make use of the Team foundation build environment variables to get key bits of data that you need for your build process logic.

I've created a PowerShell module that contains the functions that I need to build and publish a nuget package.

$ErrorActionPreference = "Stop"

Function Publish-NugetPackage
{
    param
    (
        [Parameter(Mandatory = $true, Position = 0)]
        [ValidateNotNullOrEmpty()]
        [string] $Project
    )
    # Read the product version from the assembly
    $version = Get-ChildItem -Filter "$($Project).dll" `
                             -Recurse `
                             -Path (Join-Path $env:TF_BUILD_BUILDDIRECTORY bin) `
                             -File | select -First 1 | select -expand VersionInfo | select -expand ProductVersion

    # Read the contents of the nuspec file
    $nuspecFile = Get-ChildItem -Filter "$($Project).nuspec" `
                                -Recurse `
                                -Path $env:TF_BUILD_SOURCESDIRECTORY -File | select -First 1

    $nuspecContents = Get-Content $nuspecFile.FullName

    # Replace the version of the nuspec file with the assembly version
    Set-Content -Path $nuspecFile.FullName `
                -Value ($nuspecContents -replace "[0-9]+(\.([0-9]+|\*)){1,3}", "$($version)
") `
                -Force
    # Create the nuget package
    New-NugetPackage $nuspecFile.FullName

    # Fetch the newly created nuget package
    $nugetPackage = Get-ChildItem -Filter "*.nupkg" `
                                  -Recurse `
                                  -Path $env:TF_BUILD_SOURCESDIRECTORY -File | select -First 1

    # Publish the nuget package to nuget repository
    Push-NugetPackage $nugetPackage.FullName

}

Function New-NugetPackage
{
    param
    (
        [Parameter(Mandatory = $true, Position = 0)]
        [ValidateNotNull()]
        [ValidateScript({Test-Path $_})]
        [String] $NuspecFile,

        [Parameter(Position= 1)]
        [ValidateScript({Test-Path $_ -PathType Leaf -Include "nuget.exe"})]
        [String] $NugetLocation = (Join-Path $PSScriptRoot "nuget.exe")
    )

    $command = $NugetLocation + " pack $($NuspecFile)"

    Invoke-Expression $command
}

Function Push-NugetPackage
{
     param
    (

        [Parameter(Position= 3)]
        [string] $Source = "http://prajeeshnuget.azurewebsites.net/nuget",

        [Parameter(Mandatory = $true, Position = 0)]
        [ValidateNotNull()]
        [ValidateScript({Test-Path $_})]
        [string] $PackagePath,
       
        [Parameter(Position= 2)]
        [string] $APIKey = "MY_NUGET_APIKEY",

        [Parameter(Position= 1)]
        [ValidateScript({Test-Path $_ -PathType Leaf -Include "nuget.exe"})]
        [String] $NugetLocation = (Join-Path $PSScriptRoot "nuget.exe")
    )

    $command = $NugetLocation + " push $($PackagePath) $($APIKey)"
    if(-not ([String]::IsNullOrEmpty($Source)))
    {
        $command += " -s "  + $Source
    }

    Invoke-Expression $command | Out-Null
}

Export-ModuleMember -Function Publish*

The module contains logic to update the NuSpec file with the latest product version from the assembly that is build. Once updated this file will be used to create a nuget package and then published to the repository. As you can see, I’ve used the TF_BUILD_SOURCESDIRECTORY and TF_BUILD_BUILDDIRECTORY to get the source location and build location respectively in the module. To read more about TF_BUILD variables, refer to the link https://msdn.microsoft.com/en-us/library/hh850448.aspx.

I’ve created a script that I use to invoke the functions in this module

param
(
       [string]$Project
)

$exitCode = 0
trap
{
    $Host.UI.WriteErrorLine($error[0].Exception.Message)
    $Host.UI.WriteErrorLine($error[0].Exception.StackTrace)
    if ($exitCode -eq 0)
    {
        $exitCode = 1
    }
}


Import-Module .\TFSNugetExtensions.psm1
Publish-NugetPackage $Project
exit $exitCode

The next step is to publish these scripts along with the Nuget.exe file to the source control, so that I can refer these files as part of my build definition. I’ve added these files into the BuildExtensions folder in my team project

Now we can go ahead and create the build definition for our project. While creating the build definition it’s important to create the source settings mapping for the scripts folder so that they are also available as part of the build process.


In the process tab, you can now call the script which we have created as part of the post build execution phase.



That’s it, next time you queue a build, you can see that the build creates and uploads a nuget package to the repository automatically.

Monday, July 27, 2015

DevOps with Azure resource manager - Creating your infrastructure using JSON templates and PowerShell


Azure resource group is a container that holds all the related resources for an application. The resource group could include all of the resources for an application, or only those resources that are logically grouped together. With the proper use of Azure resource manager you can now decide how you want to allocate resources to resource groups based on what makes the most sense for your organization.

Azure resource manager supports specifying the template in JSON format that defines the deployment and configuration of your machines/ application. This provides you the ease of repeatedly deploying your infrastructure in a consistent state. It also makes it easy for defining your infrastructure as a configuration instead of coding it. Dependencies are also handled easily and in an ordered way compared to the coded approach of defining the infrastructure.

In this post we’ll see how to use Azure PowerShell to create and manage a resource group and add a virtual machine to the group. Please note that you need to have PowerShell version 3.0 or above and Azure PowerShell version 0.9.0 available to use the resource manager cmdlets.

You can check the PowerShell version by typing $PSVersionTable in PowerShell console

PS C:\> $PSVersionTable

Name                           Value                                                                                                                   
----                           -----                                                                                                                    
PSVersion                      5.0.10105.0                                                                                                             
WSManStackVersion              3.0                                                                                                                      
SerializationVersion           1.1.0.1                                                                                                                 
CLRVersion                     4.0.30319.0                                                                                                             
BuildVersion                   10.0.10105.0                                                                                                             
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}                                                                                                 
PSRemotingProtocolVersion      2.3    

To check the Azure PowerShell version use the cmdlet Get-Module Azure | select Version on the console
PS C:\> Get-Module Azure | select version

Version
-------
0.9.3  

Before starting to create the resource group, we need to add the Azure account to PowerShell session using the Add-AzureAccount cmdlet

Add-AzureAccount

Note: If you have multiple subscriptions available you need to select a subscription using the Select-AzureSubscription cmdlet.

Azure PowerShell installation includes more than one PowerShell module. To use the Azure resource manager module, we must explicitly switch to the resource manager module by using the Switch-AzureMode cmdlet as given below.

Switch-AzureMode AzureResourceManager

Next step is to create the JSON file to use as a template. For our sample I;ve used the same one from Microsoft Azure website as given below.

{http://www.bodurov.com/JsonFormatter/images/Expanded.gif
    "$schema": "http://schema.management.azure.com/schemas/2014-04-01-preview/deploymentTemplate.json#",
    "contentVersion": "1.0.0.0",
    "parameters": {http://www.bodurov.com/JsonFormatter/images/Expanded.gif
        "storageAccountName": {http://www.bodurov.com/JsonFormatter/images/Expanded.gif
            "type": "string",
            "metadata": {http://www.bodurov.com/JsonFormatter/images/Expanded.gif
                "Description": "Unique DNS Name for the Storage Account where the Virtual Machine's disks will be placed."
            }
        },
        "username": {http://www.bodurov.com/JsonFormatter/images/Expanded.gif
            "type": "string",
            "metadata": {http://www.bodurov.com/JsonFormatter/images/Expanded.gif
                "Description": "Username for the Virtual Machine."
            }
        },
        "password": {http://www.bodurov.com/JsonFormatter/images/Expanded.gif
            "type": "securestring",
            "metadata": {http://www.bodurov.com/JsonFormatter/images/Expanded.gif
                "Description": "Password for the Virtual Machine."
            }
        },
        "dnsName": {http://www.bodurov.com/JsonFormatter/images/Expanded.gif
            "type": "string",
            "metadata": {http://www.bodurov.com/JsonFormatter/images/Expanded.gif
                "Description": "Unique DNS Name for the Public IP used to access the Virtual Machine."
            }
        },
        "osVersion": {http://www.bodurov.com/JsonFormatter/images/Expanded.gif
            "type": "string",
            "defaultValue": "2012-R2-Datacenter",
            "allowedValues": [http://www.bodurov.com/JsonFormatter/images/Expanded.gif
                "2008-R2-SP1",
                "2012-Datacenter",
                "2012-R2-Datacenter",
                "Windows-Server-Technical-Preview"
            ],
            "metadata": {http://www.bodurov.com/JsonFormatter/images/Expanded.gif
                "Description": "The Windows version for the VM. This will pick a fully patched image of this given Windows version. Allowed values: 2008-R2-SP1, 2012-Datacenter, 2012-R2-Datacenter, Windows-Server-Technical-Preview."
            }
        }
    },
    "variables": {http://www.bodurov.com/JsonFormatter/images/Expanded.gif
        "location": "West Europe",
        "imagePublisher": "MicrosoftWindowsServer",
        "imageOffer": "WindowsServer",
        "OSDiskName": "osdiskforwindowssimple",
        "nicName": "armDemoVM01",
        "addressPrefix": "10.0.0.0/16",
        "subnetName": "Subnet",
        "subnetPrefix": "10.0.0.0/24",
        "storageAccountType": "Standard_LRS",
        "publicIPAddressName": "armDemoPublicIP01",
        "publicIPAddressType": "Dynamic",
        "vmStorageAccountContainerName": "vhds",
        "vmName": "ARMDemoVM01",
        "vmSize": "Standard_D1",
        "virtualNetworkName": "ARMDEMOVNET01",
        "vnetID": "[resourceId('Microsoft.Network/virtualNetworks',variables('virtualNetworkName'))]",
        "subnetRef": "[concat(variables('vnetID'),'/subnets/',variables('subnetName'))]"
    },
    "resources": [http://www.bodurov.com/JsonFormatter/images/Expanded.gif
        {http://www.bodurov.com/JsonFormatter/images/Expanded.gif
            "type": "Microsoft.Storage/storageAccounts",
            "name": "[parameters('storageAccountName')]",
            "apiVersion": "2015-05-01-preview",
            "location": "[variables('location')]",
            "properties": {http://www.bodurov.com/JsonFormatter/images/Expanded.gif
                "accountType": "[variables('storageAccountType')]"
            }
        },
        {http://www.bodurov.com/JsonFormatter/images/Expanded.gif
            "apiVersion": "2015-05-01-preview",
            "type": "Microsoft.Network/publicIPAddresses",
            "name": "[variables('publicIPAddressName')]",
            "location": "[variables('location')]",
            "properties": {http://www.bodurov.com/JsonFormatter/images/Expanded.gif
                "publicIPAllocationMethod": "[variables('publicIPAddressType')]",
                "dnsSettings": {http://www.bodurov.com/JsonFormatter/images/Expanded.gif
                    "domainNameLabel": "[parameters('dnsName')]"
                }
            }
        },
        {http://www.bodurov.com/JsonFormatter/images/Expanded.gif
            "apiVersion": "2015-05-01-preview",
            "type": "Microsoft.Network/virtualNetworks",
            "name": "[variables('virtualNetworkName')]",
            "location": "[variables('location')]",
            "properties": {http://www.bodurov.com/JsonFormatter/images/Expanded.gif
                "addressSpace": {http://www.bodurov.com/JsonFormatter/images/Expanded.gif
                    "addressPrefixes": [http://www.bodurov.com/JsonFormatter/images/Expanded.gif
                        "[variables('addressPrefix')]"
                    ]
                },
                "subnets": [http://www.bodurov.com/JsonFormatter/images/Expanded.gif
                    {http://www.bodurov.com/JsonFormatter/images/Expanded.gif
                        "name": "[variables('subnetName')]",
                        "properties": {http://www.bodurov.com/JsonFormatter/images/Expanded.gif
                            "addressPrefix": "[variables('subnetPrefix')]"
                        }
                    }
                ]
            }
        },
        {http://www.bodurov.com/JsonFormatter/images/Expanded.gif
            "apiVersion": "2015-05-01-preview",
            "type": "Microsoft.Network/networkInterfaces",
            "name": "[variables('nicName')]",
            "location": "[variables('location')]",
            "dependsOn": [http://www.bodurov.com/JsonFormatter/images/Expanded.gif
                "[concat('Microsoft.Network/publicIPAddresses/', variables('publicIPAddressName'))]",
                "[concat('Microsoft.Network/virtualNetworks/', variables('virtualNetworkName'))]"
            ],
            "properties": {http://www.bodurov.com/JsonFormatter/images/Expanded.gif
                "ipConfigurations": [http://www.bodurov.com/JsonFormatter/images/Expanded.gif
                    {http://www.bodurov.com/JsonFormatter/images/Expanded.gif
                        "name": "ipconfig1",
                        "properties": {http://www.bodurov.com/JsonFormatter/images/Expanded.gif
                            "privateIPAllocationMethod": "Dynamic",
                            "publicIPAddress": {http://www.bodurov.com/JsonFormatter/images/Expanded.gif
                                "id": "[resourceId('Microsoft.Network/publicIPAddresses',variables('publicIPAddressName'))]"
                            },
                            "subnet": {http://www.bodurov.com/JsonFormatter/images/Expanded.gif
                                "id": "[variables('subnetRef')]"
                            }
                        }
                    }
                ]
            }
        },
        {http://www.bodurov.com/JsonFormatter/images/Expanded.gif
            "apiVersion": "2015-05-01-preview",
            "type": "Microsoft.Compute/virtualMachines",
            "name": "[variables('vmName')]",
            "location": "[variables('location')]",
            "dependsOn": [http://www.bodurov.com/JsonFormatter/images/Expanded.gif
                "[concat('Microsoft.Storage/storageAccounts/', parameters('storageAccountName'))]",
                "[concat('Microsoft.Network/networkInterfaces/', variables('nicName'))]"
            ],
            "properties": {http://www.bodurov.com/JsonFormatter/images/Expanded.gif
                "hardwareProfile": {http://www.bodurov.com/JsonFormatter/images/Expanded.gif
                    "vmSize": "[variables('vmSize')]"
                },
                "osProfile": {http://www.bodurov.com/JsonFormatter/images/Expanded.gif
                    "computername": "[variables('vmName')]",
                    "adminUsername": "[parameters('username')]",
                    "adminPassword": "[parameters('password')]"
                },
                "storageProfile": {http://www.bodurov.com/JsonFormatter/images/Expanded.gif
                    "imageReference": {http://www.bodurov.com/JsonFormatter/images/Expanded.gif
                        "publisher": "[variables('imagePublisher')]",
                        "offer": "[variables('imageOffer')]",
                        "sku": "[parameters('osVersion')]",
                        "version": "latest"
                    },
                    "osDisk": {http://www.bodurov.com/JsonFormatter/images/Expanded.gif
                        "name": "osdisk",
                        "vhd": {http://www.bodurov.com/JsonFormatter/images/Expanded.gif
                            "uri": "[concat('http://',parameters('storageAccountName'),'.blob.core.windows.net/',variables('vmStorageAccountContainerName'),'/',variables('OSDiskName'),'.vhd')]"
                        },
                        "caching": "ReadWrite",
                        "createOption": "FromImage"
                    }
                },
                "networkProfile": {http://www.bodurov.com/JsonFormatter/images/Expanded.gif
                    "networkInterfaces": [http://www.bodurov.com/JsonFormatter/images/Expanded.gif
                        {http://www.bodurov.com/JsonFormatter/images/Expanded.gif
                            "id": "[resourceId('Microsoft.Network/networkInterfaces',variables('nicName'))]"
                        }
                    ]
                }
            }
        }
    ]
}
Top of Form
Bottom of Form

Save this file to a JSON file on the disk. In this sample I’ve saved the file as armdemo01.json.
Next we have to create a Azure resource group using the New-AzureResourceGroup cmdlet as given below.

$location="West Europe"
$resourceGroupName = "ARMRGDemo01"
New-AzureResourceGroup -Name $resourceGroupName -Location $location

PS C:\PSScripts> New-AzureResourceGroup -Name $resourceGroupName -Location $location


ResourceGroupName : ARMRGDemo01
Location          : westeurope
ProvisioningState : Succeeded
Tags              :
Permissions       :
                    Actions  NotActions
                    =======  ==========
                    *                 
                   
ResourceId        : /subscriptions/2a9f4b1c-91fd-4fe6-8f9a-fd2ea8aa4840/resourceGroups/ARMRGDemo01

By opening the management portal we can now verify the resource group created successfully.



After creating the resource group, we can now use the template to manage the resources in the resource group. To create the deployment group, run the New-AzureResourceGroupDeployment cmdlet as given below.

$resourceGroupName = "ARMRGDemo01"
$templateFile = "C:\PSScripts\armdemo01.json"
New-AzureResourceGroupDeployment -Name "ARMDemoDeployment01" -ResourceGroupName $resourceGroupName -TemplateFile $templateFile

PS C:\PSScripts> New-AzureResourceGroupDeployment -Name "ARMDemoDeployment01" -ResourceGroupName $resourceGroupName -TemplateFile $templateFile
cmdlet New-AzureResourceGroupDeployment at command pipeline position 1
Supply values for the following parameters:
(Type !? for Help.)
storageAccountNameFromTemplate: armstrgdemo01
username: prajeesh
dnsName: armcsdemo01


DeploymentName    : ARMDemoDeployment01
ResourceGroupName : ARMRGDemo01
ProvisioningState : Succeeded
Timestamp         : 7/27/2015 5:32:04 PM
Mode              : Incremental
TemplateLink      :
Parameters        :
                    Name             Type                       Value    
                    ===============  =========================  ==========
                    storageAccountName  String                     armstrgdemo01
                    username         String                     prajeesh 
                    password         SecureString                        
                    dnsName          String                     armcsdemo01
                    osVersion        String                     2012-R2-Datacenter
                   
Outputs           : 

During the execution, the command will prompt for some parameters as given in the example above, provide values for the storage account name, virtual machine username and password to proceed.

After successful execution, we can now verify the details on the management portal to see the resources created in the group.