Showing posts with label Azure. Show all posts
Showing posts with label Azure. Show all posts

Tuesday, January 21, 2020

My top pros and cons of Azure IAAS


Azure, Microsoft’s public cloud computing platform as it promises provides a wide range of cloud services including those for computing, analytics, storage and networking. There are a lot written about the offerings from Azure including  IAAS, PAAS and SAAS solutions, and how to pick and choose the best offerings that suites your problem domain.
I’ve initially considered tweeting about this content as a series of small tweets, but I also wanted to keep this as an educational or informative content for other fellow developers and architects who would also be using IAAS for similar scenarios. So this is about my experiences with a lift and shift scenario, where our focus was to move the on-prem customer solution to a hybrid cloud infra. I have to say that with the wide range of tools and solutions from Azure, it was not only effortless but also to a much advanced, secure and better infrastructure. But there are some obvious things that I missed during this migration phase.

Below are my top pros and cons of Azure IAAS


Updating the virtual machines that don’t have outgoing internet access is a pain.


For confidentiality and data integrity purposes we had to prevent incoming and outgoing internet access from our virtual networks. We achieved this by configuring network security groups and defining allow/ deny rules for all incoming and outgoing traffic within the subnets. Now Microsoft provides something called as service tags which represents a group of IP address prefixes from a given Azure service to the virtual networks. You can make use of these service tags to create exception rules to the NSGs. For e.g you can block all incoming traffic to the VMs except from the Azure Storage services or Vault.
It is a very good service from MS and by making use of these service tags you don’t have to worry about the changes in the IP address of the Azure data center or any other services provided by Microsoft.
The problem was with windows updates on the VMs. With outgoing internet blocked from the virtual machines, our VMs were not able to contact update center to download critical updates/ patches to be applied. This creates a security issue and there is no information on the internet or from Microsoft support on the IP addresses to whitelist for allowing automatic updates. When the Azure support was contacted for this issue, they point you to a planned task to implement this feature which is in progress from 2017.
Now you are left with 2 options, either to create and configure a WSUS server on Azure or make use of the new Update Management feature (which you have to pay for 😊)


Azure monitoring needs access to a storage account in South Central US, or else it will not receive heartbeats from your Virtual machines.


This again relates to the previous scenario, were we need to block traffic to all external locations except West Europe to ensure that our data does not leave Europe. But there is a mysterious storage account created in our subscription which even the subscription owner cannot see (this we were aware only after contacting the MS support via CSP), that needs to be contacted to properly run the Health Service on the VMs. You need to explicitly allow access to the IP address of the domain “scadvisorcontent.blob.core.windows.net” for monitoring to work properly.
Till now Azure Support or Microsoft does not have an answer to this suspicious storage account!!

Virtual machines with private IP address that are in the backend pool of a private load balancer cannot access internet.


To summarize, if you want internet access from a virtual machine without a public IP address, you need to associate the VM that is behind a Standard ILB, to a Public IP. The dirty workaround is to create a dummy load balancer with a public IP address and use the VMs in the backend pool of this load balancer.

AzureDevOps does not have a service tag.



We use Azure DevOps extensively at work for planning, coding, building, testing and releasing software on all our environments, and I have to say that this is one of the best products from Microsoft. It is easy to customize, user friendly and have loads of features for all stages of your application development life cycle management.
Unfortunately Azure still doesn’t provide a service tag for Azure DevOps services and you have to go through the painful process of finding out the IP address ranges of these services and adding custom rules to the NSGs.
Not too bad as the update servers, Microsoft atleast has some documentation on these IP addresses. (https://docs.microsoft.com/en-us/azure/devops/organizations/security/allow-list-ip-url?view=azure-devops)

Azure monitoring is a great solution, but can be improved.


Azure Monitor helps you understand how your applications are performing and proactively identifies issues affecting them and the resources they depend on. It’s an umbrella of services that gives you a way to collect, analyze, and act on all the telemetry from your cloud and on-premises environments. Even though it’s very easy to setup and configure compared to the other services, the visualizations and UI can be improved a lot. I do miss the drill down capabilities from the widgets and graphs added to the dashboard. You can get this working by making use of Power BI or workbooks in the dashboards, but that brings the additional complexities into the service.

Now that we have seen the negatives, I’d like to end this post with the reasons why I’m choosing Azure for my next project as well.


Azure support is great.


They are just fantastic. Fast to respond, always willing to help, trying to find out solutions, providing detailed and updated documentation, free and practical learning paths, hands-on-labs. Contact the support anytime, and you’ll not be disappointed. I’ve noticed that the maximum response time in 2 minutes in Twitter for any queries. That’s simply amazing. Kudos to the spirit and passion @AzureSupport.

Security Center is a complete perfected platform.


Azure Security Center provides unified infrastructure security management that strengthens security posture and provides advanced threat protection across your workloads running in Azure, on-premises, and in other clouds. It is an incredible tool to manage all aspects of security in the Azure Cloud. It’s very easy to create and configure policies, connect to the SIEM solution (Azure Sentinel) and protect your workloads from all kind of attacks and vulnerabilities. You don’t need to be a security expert to effectively configure your infrastructure.
As a CTO/ CIO of a company the tool provides all kind of information and trust that you need from a security solution.

Simple, extensive and up-to-date documentation


The best documentation on any cloud service provider you can find in the internet. Do I need to say more 😊

Thursday, June 7, 2018

My new course : Continuous Delivery with Visual Studio Team Services published


Over the last 3 months I was busy working on my course on Continuous Delivery with Visual Studio Team Services. 

The course is published by Packt - a leading UK provider of Technology eBooks, Videos and Blogs, and their editors and marketing department has done a fantastic job of helping me with with creative graphics and animations to bring life to this course.


This course is intended for software development professionals who want to learn about Continuous Delivery in a technical value stream as supported by Visual Studio Team Services in order to continually deliver working software at scale. You'll learn how to create, build, test and release a containerized ASP.NET web api to kubernetes cluster on Azure.


The course consists out of seven sections:
  • Introduction to continuous delivery - We'll learn about Continuous Delivery, the business goals and principles of Continuous Delivery
  • Introduction to visual studio team services - We'll learn about the different elements of Visual Studio Team Services and how they function and how using these connected tools you can move your organization to a full-fledged continuous delivery implementation.
  • Setting up the development environment for .net core applications using Docker:
  • Source code management in VSTS : We'll have a look into the Git version control in VSTS and discuss branching strategies like trunk based development and how these concepts relate to continuous delivery.
  • Build management in VSTS : We'll see how to setup a Continuous integration build in VSTS that helps the code stay robust enough that team members can fearlessly make changes to the code base without worrying about breaking existing functionality.
  • Continuous testing : This section we’ll learn how to ensure that an application's components function correctly when assembled together. We'll see how to automate the tests and include them in your Continuous Integration scheme to get the most value from the tests.
  • Release management in VSTS : This section shows you how to do automatic deployments by using Release Management. You'll learn Release Management in VSTS, how it works and how to use release management to deploy our sample application to a Kubernetes cluster on Azure.

The course follows a phased approach to continuous delivery to automate build, deploy and management of .net core applications to Azure Container Services using VSTS ALM. By the end of this course, you'll have the necessary knowledge to create a fully automated build and release pipeline using Visual Studio Team Services for your .NET applications. And don't forget to check the course at https://www.packtpub.com/virtualization-and-cloud/continuous-delivery-visual-studio-team-services-video

Monday, September 28, 2015

Docker for windows on Azure VM : Securing the host and TLS

In the previous post about docker on windows, we looked into the details of creating a Windows 2016 TP3 VM in Azure and looked into the details of managing containers on the host VM that is running the Docker daemon. This post we’ll look into the security considerations when running Docker and how to secure the Docker daemon with TLS. We’ll be using openssl to create and manage certificates for SSL. 

If you don’t have openssl installed on the machines, download the binaries for windows from the location http://gnuwin32.sourceforge.net/packages/openssl.htm.

To setup TLS for Docker, we need to follow the below given steps.
  1. Create certificate authority (CA)
  2. Setup the server private key
  3. Create certificate signing request for the server (CSR)
  4. Sign the server key with the CSR against the CA
  5. Create client private key and CSR
  6. Sign the client key with the CSR against the CA
  7. Copy the server certificates to the docker host machine
  8. Add firewall rule for allowing communication to port 2376

Before we start using the openssl executable, we need to ensure that the configuration file for openssl is available. The version 1.0 of OpenSSL requires an "openssl.cnf" configuration file. Openssl reads the location of the configuration file by using the environment variable OPENSSL_CONF. For this example, we can download and use the configuration file from https://www.tbs-certificates.co.uk/openssl-dem-server-cert-thvs.cnf.

Creating the certificates.

Download and extract the openssl binaries from location http://gnuwin32.sourceforge.net/packages/openssl.htm to the C:\OpenSSL folder

  • Download and copy the openssl configuration file to C:\OpenSSL\openssl.cnf file
  • Setup the environment variable for opensssl configuration

param
(
       [string] $Path = "C:\OpenSSL",
       [string] $CertLocation = "C:\Docker\Certs"
)
$opensslExe = Join-Path $Path "openssl.exe"
$opensslCnf = Join-Path $Path "openssl.cnf"

if(-not (Test-Path $opensslExe -ErrorAction SilentlyContinue))
{
       throw "openssl.exe not found at location $Path"
}

$env:OPENSSL_CONF= $opensslCnf

  • During certificate generation, there is an .rnd file that OpenSSL needs to write to. We need to set the RANDFILE environment variable to a directory at the certificates location

$env:RANDFILE = Join-Path $CertLocation ".rnd"

  • First we need to create the certificate authority private key

& $opensslExe genrsa -aes256 -out ca-key.pem 2048


  • Using the CA private key, create the CA certificate

& $opensslExe req -new -x509 -days 365 -key ca-key.pem -subj "/C=NL/ST=UT/L=Amersfoort/O=Prajeesh" -sha256 -out ca.pem

  • Next we’ll create the server private key

& $opensslExe genrsa -aes256 -out server-key.pem 2048


  • After this we need to create the certificate signing request (CSR) for the server key. Use the host server IP address while creating the server key

& $opensslExe req -subj "/C=NL/ST=UT/L=Amersfoort/O=Prajeesh" -new -key server-key.pem -out server.csr

  • Before we sign the server key we need to define the certificate extension to specify the subjectAltName. The subjectAltName allows us to specify things such as the IP addresses we will allow connections on.

"subjectAltName = IP:10.10.10.20,IP:127.0.0.1,DNS.1:*.cloudapp.net,DNS.2:*.*.cloudapp.azure.com" | Out-File extfile.cnf -Encoding ASCII

  • Now we can sign the server key

& $opensslExe x509 -req -days 365 -in server.csr -CA ca.pem -CAkey ca-key.pem -CAcreateserial -out server-cert.pem -extfile extfile.cnf


  • Next we’ll create the client keys. First we’ll create the client’s private key

& $opensslExe genrsa -out client-key.pem 2048


  • Then we’ll create the client certificate signing request

& $opensslExe req -subj "/CN=client" -new -key client-key.pem -out client.csr

  • To sign the client key we need an extensions config file with the extendedKeyUsage extension in order to make the key suitable for client authentication

"extendedKeyUsage = clientAuth" | Out-File extfile.cnf -Encoding ASCII

  • Now we can sign the client key

& $opensslExe x509 -req -days 365 -in client.csr -CA ca.pem -CAkey ca-key.pem -CAcreateserial -out client-cert.pem -extfile extfile.cnf

Configuring the docker host

  • We’ve successfully created all the certificates needed for our setup. Now we need to copy the server certificates to the VM where the docker daemon is running. You can either use AzCopy utility to copy the certificates that are added to the container as blob or download the certificates from an Uri using Invoke-WebRequest cmdlet in the machine. The server certificates should be copied to the directory C:\ProgramData\Docker\certs.d folder



  • Next we have to open the connections to port 2376 using the New-NetFirewallRule cmdlet as given below.



  • Restart the docker service on the host
  • Copy the client certificates to the .docker folder in the $Env:USERPROFILE folder.

Connecting using TLS

  • Now connect to docker host using the ip address of the host and port 2376 using –tlsverify option from the client machine


docker --tlsverify -H tcp://.westeurope.cloudapp.azure.com:2376 ps -a


Sunday, August 30, 2015

Containerize your applications using docker for Windows - Part 1

Containers in software terminology is a lightweight virtual environment that groups and isolates a set of processes and resources such as memory, CPU, disk etc. from the host and any other containers. They include the application and all of its dependencies, but share the kernel with the other containers. One huge benefit of containers are that they are not tied to any specific infrastructure. The below diagram depicts docker containers on an infrastructure.



 Containers help developers create and test applications in their local environment and later containerize the application, which creates a docker image with the app and the components required to run the app which is later used to create a container in an environment which is needed to host the container.

Because the container has everything it needs to run your application, they are very portable and can run on any machine that is running Windows Server 2016. You can create and test containers locally, then deploy that same container image to your company's private cloud, public cloud or service provider. The natural agility of Containers supports modern app development patterns in large scale, virtualized and cloud environments.

With containers, developers can build an app in any language. These apps are completely portable and can run anywhere - laptop, desktop, server, private cloud, public cloud or service provider - without any code changes eventually helping developers build and ship higher-quality applications, faster. For more details on Windows Server Containers you can refer to this article in MSDN.

In this series of posts we’ll see, how to create virtual machines on Azure and later using these to host containers to deploy and test applications.

To start with we’ll create a VM in a resource group and use this to create and host our containers. To create and host containers in a VM, you need a VM with Windows Server 2016 TP3 or later with the windows server containers feature. I’ve created a PowerShell module to complete the process.

function New-AzureVMInRG
{
       [CmdletBinding()]
       param
       (
              [Parameter(Mandatory=$true)]
              [ValidateNotNullOrEmpty()]
              [string] $ResourceGroupName,

              [Parameter(Mandatory=$true)]
              [ValidateNotNullOrEmpty()]
              [ValidateScript({($_.Length -ge 3 -and $_.Length -lt 24) -and (-not($_ -cmatch '[A-Z]'))})]
              [string] $StorageAccountName,

              [Parameter(Mandatory=$true)]
              [ValidateNotNullOrEmpty()]
              [string] $VNetName,

              [Parameter(Mandatory=$true)]
              [ValidateNotNullOrEmpty()]
              [string] $NICName,

              [Parameter(Mandatory=$true)]
              [ValidateNotNullOrEmpty()]
              [string] $VMName,

              [Parameter(Mandatory=$false)]
              [string] $VMSize = 'Basic_A0',

              [Parameter(Mandatory=$false)]
              [string] $Location = 'West Europe',

              [Parameter(Mandatory=$false)]
              [string] $DNSDomainNameLabel,

              [Parameter(Mandatory=$false)]
              [string] $AvailabilitySetName
       )

       EnsureResourceGroup $ResourceGroupName $Location -Verbose
       $storageAccount = EnsureStorageAccount $ResourceGroupName $StorageAccountName -Verbose
       EnsureVirtualNetwork $VNetName $ResourceGroupName $Location -Verbose

       if([string]::IsNullOrWhiteSpace($DNSDomainNameLabel))
       {
              $DNSDomainNameLabel = $ResourceGroupName.ToLower()
       }
       $publicIP = EnsurePublicIPAddress $NICName $DNSDomainNameLabel $ResourceGroupName $Location
       $nic = EnsureAzureNetworkInterface $NICName $publicIP $VNetName $ResourceGroupName $Location
      
       $availabilitySet = EnsureAvailabilitySet -$AvailabilitySetName $ResourceGroupName $Location

       $vmConfig = New-AzureVMConfig -VMName $VMName -VMSize $VMSize -AvailabilitySetId $availabilitySet.Id
       $vmConfig = New-AzureVMConfig -VMName $VMName -VMSize $VMSize
       $credentials = Get-Credential -Message "Provide the name and password for the local administrator on the virtual machine."
       $vmConfig = Set-AzureVMOperatingSystem -VM $vmConfig -Windows -ComputerName $VMName -Credential $credentials -ProvisionVMAgent -EnableAutoUpdate
       $vmConfig = Set-AzureVMSourceImage -VM $vmConfig -PublisherName "MicrosoftWindowsServer" -Offer "WindowsServer" -Skus "2016-Technical-Preview-3-with-Containers"
       $vmConfig = Add-AzureVMNetworkInterface -VM $vmConfig -Id $nic.Id

       $osDiskUri = $storageAccount.PrimaryEndpoints.Blob.ToString() + "vhds/" + $VMName + "OSDisk.vhd"
       $vmConfig = Set-AzureVMOSDisk -VM $vmConfig -Name "OSDisk" -VhdUri $osDiskUri -CreateOption fromImage
       New-AzureVM -ResourceGroupName $ResourceGroupName -Location $Location -VM $vmConfig
}

You can download the code from GitHub url : AzureResourceGroupExtensions.psm1
Using the module you can create a VM in an Azure resource group as given below.

New-AzureVMInRG -ResourceGroupName "DockerDemo01" -StorageAccountName "dockerdemostrg01" -VNetName "DDVNet01" -NICName "DDNic01" -VMName "DDemoVM01" -DNSDomainNameLabel "ddemodomain01" -AvailabilitySetName "DDemoSet01"

Once the VM is created, we can login to the VM to create a container and test the settings. To login, we need to first download the remote desktop connection file. To download the .RDP file, I’ve created a module member as given below.

function Get-VMRemoteDesktopFile
{
       param
       (
              [Parameter(Mandatory=$true, Position = 0)]
              [Parameter(ParameterSetName = "VMName")]
              [string] $VMName,
             
              [Parameter(Mandatory=$true, Position = 0, ValueFromPipeline = $true)]
              [Parameter(ParameterSetName = "VM")]
              [Microsoft.Azure.Commands.Compute.Models.PSVirtualMachine] $VM,

              [Parameter(Mandatory=$true, Position = 1)]
              [ValidateNotNullOrEmpty()]
              [ValidateScript({(Split-Path $_ -leaf).EndsWith("rdp") })]
              [string] $Path
       )
       $folder = Split-Path -Path $Path -Parent
       if(-not (Test-Path $folder))
       {
              New-Item -Path $folder -ItemType Directory -Force | Out-Null
       }

       if($PSCmdlet.ParameterSetName -eq "VMName")
       {
              Get-AzureVM |? {$_.Name -eq $VMName} | Get-AzureRemoteDesktopFile -LocalPath $Path
       }
       else
       {
              $VM | Get-AzureRemoteDesktopFile -LocalPath $Path
       }
}

Once you have the .rdp file download using the function given above, you can login to the virtual machine and then create containers.

In the windows administration console, start a PowerShell session by typing PowerShell. The command prompt will change to PS indicating a valid PowerShell session


You can create a new container using the New-Container cmdlet. The cmdlet needs a ContainerImage and a SwitchName as parameters.



Using these options, we can create a new container as given below.

$containerImage = Get-ContainerImage
$vmSwitch = Get-VMSwitch | Select -ExpandProperty Name
$container = "DemoContainer01"
New-Container -Name $container -ContainerImage $containerImage -SwitchName $vmSwitch



Next we can use the Get-Container cmdlet to check the status of the container.

Get-Container

As you can see, the status of the container is Off. You can start the container using the Start-Container cmdlet:

Start-Container -Name $container



Once the container is started, you can interact with the containers using PowerShell remoting commands such as Invoke-Command, or Enter-PSSession. For e.g. you can use the Enter-PSSession cmdlet by providing the container Id to create an interactive session as given below.

$demoContainer = Get-Container -Name $container
Enter-PSSession -ContainerId $demoContainer.Id -RunAsAdministrator


Later, you can exit the session and stop the container using the Stop-Container cmdlet.



In the upcoming posts, we’ll see how to create an image of the container, install web server and deploy a website application to the container.