Showing posts with label Chef. Show all posts
Showing posts with label Chef. Show all posts

Friday, July 15, 2016

Automate your windows development machine using Chef

I was trying to build my windows workstation using automation and this time decided to use chef as the CM tool and play around with it. I have used PowerShell DSC to configure my desktop and was trying to see how Chef can be used also. Below are the steps that I followed to get it up and running. Hope this gives you an overview on the process and help you to rebuild your machine also in code!!!

Step 1: Setup the chef development kit on the machine

The first step is to install the chef development kit and we can use the powershell package manager for this.

Step 2: Create a chef repository

A chef repo is the directory structure that chef client understands and find related items when we are working with the chef client. The chef development kit has a generate command that can be used to create a repository. In the PowerShell prompt type the command

Chef generate repo chef-workstation

This will create a folder chef-workstation under the current path and provision the default files and folder structure.


You can inspect the structure of the repository using your favorite editor by opening the root folder. I’ve used visual studio code to explore the file contents


Step 3: Create our first cookbook

We can use the generate command to create a cookbook in our repository. Before running the command, switch to the cookbooks folder in your repo and type in

Chef generate cookbook windowsdev


Step 4: Add some metadata for the cookbook

In Visual studio code, open the metadata.rb file under the windowsdev cookbook folder and update the metadata as given below. 

name 'windowsdev'
maintainer 'Prajeesh Prathap'
maintainer_email 'prajeesh.prathap@infosupport.com'
license 'all_rights'
description 'configures a windows development machine for .net development'
long_description 'configures a windows development machine for .net development'
version '0.1.0'
depends 'windows'
depends 'chocolatey'

Note that I have added a dependency for the chocolatey and windows cookbook in the metadata. We will later use the resources from this cookbook to install packages from chocolatey on the node.

The windows and chocolatey cookbooks are available in the chef supermarket and will be used when the dependencies are needed.

Step 5: Create a recipe

As I mentioned in the earlier step, we will be using chocolatey to setup the packages on the machine. Before using chocolatey, we need to first ensure that chocolatey is installed on the workstation. To install chocolatey, we will create a recipe and use that in the cookbook before we apply any packages.
Navigate to the windowsdev cookbook folder and type in the command

Chef generate recipe setupchoco

Open the newly created recipe file in the editor and add the contents given below.

#
# Cookbook Name:: windowsdev
# Recipe:: setupchoco
#
# Copyright (c) 2016 Prajeesh Prathap, All Rights Reserved.

# Configures chocolatey on the machine

include_recipe 'windows'

chocolatey_path = "#{ENV['SYSTEMDRIVE']}\\ProgramData\\chocolatey\\bin"

windows_path 'update_path_for_system' do
  action :add
  path chocolatey_path
end

ruby_block 'add_chocolatey_path' do
  block do
    new_path = "#{ENV['PATH']};#{chocolatey_path}"
    ENV['PATH'] = "#{ENV['PATH']};#{new_path}"
  end

  not_if {
    (ENV['PATH'].split(';').collect { | element | element.downcase }).include? chocolatey_path.downcase
  }
end

powershell_script 'chocolatey_install' do
  code <<-eoh o:p="">
powershell -noprofile -inputformat none -noninteractive -executionpolicy bypass -command "iex ((new-object net.webclient).DownloadString('https://chocolatey.org/install.ps1'))"
EOH
 #  not_if { ChocolateyHelpers::chocolatey_installed? }
  not_if "test-path '#{chocolatey_path}\\choco.exe'"
end


The above recipe will install chocolatey on the workstation if not present.

Step 6: Setup PowerShell and WinRM

Similarly create  a new recipe and name it setuppowershell. Add the below code to the setuppowershell.rb file.

#
# Cookbook Name:: windowsdev
# Recipe:: setuppowershell
#
# Copyright (c) 2016 Prajeesh Prathap, All Rights Reserved.
powershell_script 'ExecutionPolicyUnrestricted' do
  code <<-eoh o:p="">
powershell -noprofile -executionpolicy bypass -command {set-executionpolicy unrestricted -force -scope localmachine}
exit 0
EOH
  only_if "(get-executionpolicy -scope localmachine) -ne 'unrestricted'"
end

powershell_script 'ExecutionPolicyUnrestrictedX86' do
  architecture :i386  # Handle 32-bit Powershell (no-op if OS is 32-bit)
  code <<-eoh o:p="">
powershell -noprofile -executionpolicy bypass -command {set-executionpolicy unrestricted -force -scope localmachine}
exit 0
EOH
  only_if "(get-executionpolicy -scope localmachine) -ne 'unrestricted'"
end

powershell_script 'Setup WINRM' do
  code <<-eoh o:p="">
  Set-WSManQuickConfig -Force
  EOH
  flags '-NoLogo, -NonInteractive, -NoProfile, -ExecutionPolicy Unrestricted'
end

Step 7: Add extra files or scripts

If you want to add some custom scripts and execute them as part of the run list, then you can include them in the files directory of the cookbook and refer to those in the recipes.
To add a custom powershell script to the cookbook, use the chef generate file command as given below

Chef generate file configure_psmodules.ps1



Open the script file in the editor and paste the code given below and save it. This will install some modules from the PowerShell gallery to the machine.

Install-Module -Name xPSDesiredStateConfiguration -Force
Install-Module -Name xDSCResourceDesigner -Force
Install-Module -Name PSScriptAnalyzer –Force


Step 8: Use the script file in the recipe

We can also execute the code from the script files that we included as part of the cookbook. To execute the configure_psmodules.ps1 file, we’ll now make the changes to the setuppowershell recipe and add the following lines below

cookbook_file "#{ENV['USERPROFILE']}/configure_psmodules.ps1" do
  source 'configure_psmodules.ps1'
end

powershell_script "Run code from script to install modules" do
  code "#{ENV['USERPROFILE']}/configure_psmodules.ps1"
end

This will copy the file to the users folder and then execute the file.


Step 10: Install chocolatey packages

Create a new recipe and name it chocopackages. This recipe will be used to mention all the chocolatey packages needed for the machine.

Add the code to the recipe, to install the chocolatey packages
#
# Cookbook Name:: windowsdev
# Recipe:: chocopackages
#
# Copyright (c) 2016 Prajeesh Prathap, All Rights Reserved.
include_recipe 'chocolatey'

chocolatey 'Fiddler'

chocolatey 'SublimeText2'

chocolatey '7zip.install'

chocolatey 'freemind'

chocolatey 'javaruntime'

chocolatey 'jre8'

chocolatey 'picpick.portable'

chocolatey 'adobereader'

chocolatey 'github'

chocolatey 'sysinternals'

chocolatey 'procexp'

chocolatey 'pester'

chocolatey 'beyondcompare'

chocolatey 'ilspy'

chocolatey 'ncrunch-vs2015'

chocolatey 'ProcessExplorer'


Step 11: Linking it all together

We have a list of recipes to be executed as part of the run. But before we start, we need to link all these recipes in the default.rb recipe of the cookbook. You can use the include statement to add the recipes to be executed in the cookbook. Open the default.rb file and add the below lines of code.
#
# Cookbook Name:: windowsdev
# Recipe:: default
#
# Copyright (c) 2016 Prajeesh Prathap, All Rights Reserved.
# Call all recipies from here

if node['platform'] == "windows"
  include_recipe 'windowsdev::setupchoco'
  include_recipe 'windowsdev::setuppowershell'
  include_recipe 'windowsdev::configurereg'
  include_recipe 'windowsdev::chocopackages'
end


Step 12: Upload the cookbook to chef server

To upload the cookbook to the chef server you can use the knife cookbook upload [[cookbookname]] command.

Knife cookbook upload windowsdev



Step 13: Adding a new role to the chef server with cookbook

We’ll create a new Role with name devmachine and use the recipe windowsdev in the run list. Later when we bootstrap the node, we’ll use this role so that our recipe will be used to configure the node.
To create a role file, in the code editor add a new file name devmachine.json under the roles folder in the chef repository. Open the JSON file and add the contents below.

{
    "name": "devmachine",
    "description": "Configures a windows development machine",
    "chef_type": "role",
    "json_class": "Chef::Role",
    "default_attributes": {
    },
    "override_attributes": {
    },
    "run_list": [
        "recipe[windowsdev]"
    ]
}



We’ll use the knife command to upload this role to the server.
Knife role from file .\roles\devmachine.json



You can view the role and details in the chef server also.


Step 14: Bootstrapping the node

A bootstrap is a process that installs the chef-client on a target system so that it can run as a chef-client and communicate with a Chef server. We can use the knife bootstrap command to bootstrap our node and use the devmachine role as the run list.

knife bootstrap windows winrm 127.0.0.1 --node-name ‘YOUR_NODENAME’ -r 'role[devmachine]'

Please note that you have to configure your winrm username and password in the knife.rb file to get this working.




And that's all! You are now able to setup a development machine with all the basic software and configuration you need. The final structure and the files used in this sample can be found at my Github repository (https://github.com/prajeeshprathap/chefdev) . Go ahead and play with it!!!!

Monday, August 17, 2015

Custom DSC resource to copy files from Github

PowerShell DSC enables deploying and managing configuration data for software services and environment management very easy and convenient. Developers can create and make use of custom DSC resources to automate the daily tasks performed and distribute them for usage to the community.

To make DSC resources available in your environment, you need to have the resources available in the PSModulePath location in the computer that the resource has to be executed. If you need to use a resource from a location like GitHub for e.g., you need to download the resource and copy that to the local computers module path for usage. Or you can create a resource to do the same.

I've created a DSC resource to download files from a GitHub raw file url and copy that to the Module path in the local computer that the DSC resource executes.

enum Ensure
{
    Absent
    Present
}

[DscResource()]
class xGithubModule
{
    [DsCProperty(Key)]
    [System.String] $Url

    [DsCProperty(Mandatory)]
    [System.String] $Module

    [DsCProperty(Mandatory)]
    [Ensure] $Ensure

    [DsCProperty(NotConfigurable)]
    [Boolean] $IsValid

    [xGithubModule] Get()
    {      
        $this.IsValid = $false
        $fileName = $this.Url.Substring($this.Url.LastIndexOf('/'), $this.Url.Length - ($this.Url.LastIndexOf('/')))
        $moduleFolder = Join-Path 'C:\Program Files\WindowsPowerShell\Modules' $this.Module
               
        $fileExists = Test-Path (Join-Path $moduleFolder $fileName)

        if($this.Ensure -eq [Ensure]::Present)
        {
            if($fileExists)
            {
                $this.IsValid = $true
            }
        }
        else
        {
            if(-not $fileExists)
            {
                $this.IsValid = $true
            }
        }
        return $this
    }

    [void] Set()
    { 
        $modulePath = 'C:\Program Files\WindowsPowerShell\Modules'
        $fileName = $this.Url.Substring($this.Url.LastIndexOf('/'), $this.Url.Length - ($this.Url.LastIndexOf('/')))

        $moduleFolder = Join-Path 'C:\Program Files\WindowsPowerShell\Modules' $this.Module
        $filePath = Join-Path $moduleFolder $fileName               
     
        if($this.Ensure -eq [Ensure]::Present)
        { 
            Write-Verbose "Starting download file at location $($this.Url) to $filePath"

            $webClient = New-Object System.Net.WebClient
            $uri = New-Object System.Uri $this.Url
            $webClient.DownloadFile($uri, $filePath)          
        }
        else
        {          
            Remove-Item $filePath     
        }
    }

    [bool] Test()
    {
        $result = $this.Get()

        if($this.Ensure -eq [Ensure]::Present)
        {     
            if($result.IsValid)
            {
                Write-Verbose "The file from the Url $($this.Url) already exists in the modules folder"   
                return $true               
            }
            else
            {
                Write-Verbose "The file from the Url $($this.Url) does not exists in the modules folder. This will be downloaded"   
                return $false                  
            }
        }
        else
        {
            if($result.IsValid)
            {
                Write-Verbose "The file from the Url $($this.Url) does not exists in the modules folder"   
                return $true
            }
            else
            {
                Write-Verbose "The file from the Url $($this.Url) exists in the modules folder. This will be be removed"   
                return $false                
            }
        }
    }
}


You can now integrate this with Chef, and get your resources copied for the next run also.


Sunday, August 16, 2015

Software package management using PowerShell OneGet, DSC, Chef and Chocolatey

Converting your software shared repositories to a managed approach with the use of a package management system that automates the process of installing, upgrading, configuration and removal of software packages in a consistent manner is a challenging task for IT.

The idea is to distribute software installation packages that contains additional metadata like, name, version, purpose, vendor, checksum etc. that will be used in an effective manner to manage dependencies and other details that are needed for the software to run properly. Having a package management system like this, will make easier to find, install, and remove software packages in a consistent manner.

In a windows infrastructure, you can make use of the tools like chocolatey, PowerShell package management and chef to automate and create an effective package management infrastructure that converts your infrastructure as code and thus maintaining an efficient process of software installations in your company.

Without much effort you can easily setup a chocolatey repository for your packages in Azure, and then later configure this as a source in PowerShell OneGet to install packages from this source. Later you can combine this with DSC or Chef to automatically install the packages and maintain the servers in a desired state based on the configuration.

A detailed tutorial on this series can be found at the posts from the links given below:




Using DSC, OneGet and chef to install chocolatey packages

Continuing from the previous sets of posts, we'll see how to make use of the packages available in our chocolatey repository and install them using Chef.

Installation of packages from a Chocolatey source during a chef client run can be achieved in a number of ways. We can make use of the Chocolatey cookbook that is available as part of the chef community cookbooks. If you want to make use of DSC to download and install packages, then you can use a resource to install packages using OneGet on the machine.

In this part of the series, I’ll show how we can use DSC to install packages from the OneGet source.
The first step is to ensure that WMF 5.0 is installed on the server. We can make use of the chocolatey cookbook to install PowerShell 5.0 on the server as given below.

This will ensure that the required PowerShell version is installed on the server.

Once we have WMF 5.0 installed, then we can use DSC to install packages on the server.
For installing and uninstalling packages, I’ve created a DSC resource. The resource code is as given below.

enum Ensure
{
    Absent
    Present
}

[DscResource()]
class xPackageManagement
{
    [DsCProperty(Key)]
    [System.String] $Name

    [DsCProperty(Mandatory)]
    [Ensure] $Ensure

    [DsCProperty()]
    [System.String] $Version

    [DsCProperty()]
    [System.String] $Source

    [DsCProperty(NotConfigurable)]
    [Boolean] $IsValid

    [xPackageManagement] Get()
    {       
        $this.IsValid = $false
        if(-not ([string]::IsNullOrWhiteSpace($this.Version)))
        {
            $package = Get-Package |? {($_.Name -eq $this.Name) -and ($_.Version -eq $this.Version)}
        }
        else
        {
            $package = Get-Package |? {$_.Name -eq $this.Name}
        }

        if($this.Ensure -eq [Ensure]::Present)
        {
            if($package -ne $null)
            {
                $this.IsValid = $true
            }
        }
        else
        {
            if($package -eq $null)
            {
                $this.IsValid = $true
            }
        }
        return $this  
    }

    [void] Set()
    {                  
      
        if($this.Ensure -eq [Ensure]::Present)
        {      
            if(-not ([String]::IsNullOrWhiteSpace($this.Source)))
            {
                if(-not ([string]::IsNullOrWhiteSpace($this.Version)))
                {
                    Install-Package -Name $this.Name -RequiredVersion $this.Version -Source $this.Source
                }
                else
                {
                    Install-Package -Name $this.Name -Source $this.Source
                }
            }
            else
            {
                if(-not ([string]::IsNullOrWhiteSpace($this.Version)))
                {
                    Install-Package -Name $this.Name -RequiredVersion $this.Version
                }
                else
                {
                    Install-Package -Name $this.Name
                }
            }
        }
        else
        {
            if(-not ([string]::IsNullOrWhiteSpace($this.Version)))
            {
                Get-Package |? {($_.Name -eq $this.Name) -and ($_.Version -eq $this.Version)} | Uninstall-Package -Force
            }
            else
            {
                Get-Package |? {$_.Name -eq $this.Name} | Uninstall-Package -Force
            }           
        }
    }

    [bool] Test()
    {
        $result = $this.Get()

        if($this.Ensure -eq [Ensure]::Present)
        {      
            if($result.IsValid)
            {
                if(-not ([String]::IsNullOrWhiteSpace($this.Version)))
                {
                    Write-Verbose "The pacakge $($this.Name) with version $($this.Version) already exists"    
                    return $true
                }
                else
                {
                    Write-Verbose "The pacakge $($this.Name) already exists"    
                    return $true
                }
            }
            else
            {
                if(-not ([String]::IsNullOrWhiteSpace($this.Version)))
                {
                    Write-Verbose "The pacakge $($this.Name) with version $($this.Version) does not exists. This will be installed"    
                    return $false
                }
                else
                {
                    Write-Verbose "The pacakge $($this.Name) does not exists. This will be installed"    
                    return $false
                }                    
            }
        }
        else
        {
            if($result.IsValid)
            {
                Write-Verbose "The pacakge $($this.Name) does not exists"    
                return $true
            }
            else
            {
                if(-not ([String]::IsNullOrWhiteSpace($this.Version)))
                {
                    Write-Verbose "The pacakge $($this.Name) with version $($this.Version) exists. This will be uninstalled"    
                    return $false
                }
                else
                {
                    Write-Verbose "The pacakge $($this.Name) exists. This will be uninstalled"    
                    return $false
                }                 
            }
        }
    }
}

You can also download the code from the GitHub repository https://github.com/prajeeshprathap/DSCResources

Before calling this resource, we need to first register the new package source that contains our packages. We'll make use of DSC here also.

enum Ensure
{
    Absent
    Present
}

[DscResource()]
class xChocolateySource
{
    [DsCProperty(Key)]
    [System.String] $Name

    [DsCProperty(Mandatory)]
    [System.String] $Location

    [DsCProperty(Mandatory)]
    [Ensure] $Ensure

    [DsCProperty(NotConfigurable)]
    [Boolean] $IsValid

    [xChocolateySource] Get()
    {      
        $this.IsValid = $false
        $packagesource = Get-PackageSource -ProviderName chocolatey -Name $this.Name       

        if($this.Ensure -eq [Ensure]::Present)
        {
            if($packagesource -ne $null)
            {
                $this.IsValid = $true
            }
        }
        else
        {
            if($packagesource -eq $null)
            {
                $this.IsValid = $true
            }
        }
        return $this 
    }

    [void] Set()
    {                 
     
        if($this.Ensure -eq [Ensure]::Present)
        {    
            Register-PackageSource -ProviderName Chocolatey -Name $this.Name -Location $this.Location -Force -Confirm:$true           
        }
        else
        {
            Unregister-PackageSource -Source $this.Name -ProviderName Chocolatey -Force         
        }
    }

    [bool] Test()
    {
        $result = $this.Get()

        if($this.Ensure -eq [Ensure]::Present)
        {     
            if($result.IsValid)
            {
                Write-Verbose "The pacakge source $($this.Name) already exists"   
                return $true               
            }
            else
            {
                Write-Verbose "The pacakge source $($this.Name) does not exist. This will be created"   
                return $false                  
            }
        }
        else
        {
            if($result.IsValid)
            {
                Write-Verbose "The pacakge source $($this.Name) does not exists"   
                return $true
            }
            else
            {
                Write-Verbose "The pacakge source $($this.Name) exists. This will be be removed"   
                return $false                
            }
        }
    }
}

Next we’ll create a cookbook and use this resource in the cookbook to install the packages.

Knife cookbook create packagemanagement

Edit the default.rd file to invoke the created resources


After adding the resources, we can now upload the package to the server and add it to the run list

Knife cookbook upload packagemanagement




Next chef run will execute the resource and install the package from the source.