Tuesday, January 21, 2020

My top pros and cons of Azure IAAS


Azure, Microsoft’s public cloud computing platform as it promises provides a wide range of cloud services including those for computing, analytics, storage and networking. There are a lot written about the offerings from Azure including  IAAS, PAAS and SAAS solutions, and how to pick and choose the best offerings that suites your problem domain.
I’ve initially considered tweeting about this content as a series of small tweets, but I also wanted to keep this as an educational or informative content for other fellow developers and architects who would also be using IAAS for similar scenarios. So this is about my experiences with a lift and shift scenario, where our focus was to move the on-prem customer solution to a hybrid cloud infra. I have to say that with the wide range of tools and solutions from Azure, it was not only effortless but also to a much advanced, secure and better infrastructure. But there are some obvious things that I missed during this migration phase.

Below are my top pros and cons of Azure IAAS


Updating the virtual machines that don’t have outgoing internet access is a pain.


For confidentiality and data integrity purposes we had to prevent incoming and outgoing internet access from our virtual networks. We achieved this by configuring network security groups and defining allow/ deny rules for all incoming and outgoing traffic within the subnets. Now Microsoft provides something called as service tags which represents a group of IP address prefixes from a given Azure service to the virtual networks. You can make use of these service tags to create exception rules to the NSGs. For e.g you can block all incoming traffic to the VMs except from the Azure Storage services or Vault.
It is a very good service from MS and by making use of these service tags you don’t have to worry about the changes in the IP address of the Azure data center or any other services provided by Microsoft.
The problem was with windows updates on the VMs. With outgoing internet blocked from the virtual machines, our VMs were not able to contact update center to download critical updates/ patches to be applied. This creates a security issue and there is no information on the internet or from Microsoft support on the IP addresses to whitelist for allowing automatic updates. When the Azure support was contacted for this issue, they point you to a planned task to implement this feature which is in progress from 2017.
Now you are left with 2 options, either to create and configure a WSUS server on Azure or make use of the new Update Management feature (which you have to pay for 😊)


Azure monitoring needs access to a storage account in South Central US, or else it will not receive heartbeats from your Virtual machines.


This again relates to the previous scenario, were we need to block traffic to all external locations except West Europe to ensure that our data does not leave Europe. But there is a mysterious storage account created in our subscription which even the subscription owner cannot see (this we were aware only after contacting the MS support via CSP), that needs to be contacted to properly run the Health Service on the VMs. You need to explicitly allow access to the IP address of the domain “scadvisorcontent.blob.core.windows.net” for monitoring to work properly.
Till now Azure Support or Microsoft does not have an answer to this suspicious storage account!!

Virtual machines with private IP address that are in the backend pool of a private load balancer cannot access internet.


To summarize, if you want internet access from a virtual machine without a public IP address, you need to associate the VM that is behind a Standard ILB, to a Public IP. The dirty workaround is to create a dummy load balancer with a public IP address and use the VMs in the backend pool of this load balancer.

AzureDevOps does not have a service tag.



We use Azure DevOps extensively at work for planning, coding, building, testing and releasing software on all our environments, and I have to say that this is one of the best products from Microsoft. It is easy to customize, user friendly and have loads of features for all stages of your application development life cycle management.
Unfortunately Azure still doesn’t provide a service tag for Azure DevOps services and you have to go through the painful process of finding out the IP address ranges of these services and adding custom rules to the NSGs.
Not too bad as the update servers, Microsoft atleast has some documentation on these IP addresses. (https://docs.microsoft.com/en-us/azure/devops/organizations/security/allow-list-ip-url?view=azure-devops)

Azure monitoring is a great solution, but can be improved.


Azure Monitor helps you understand how your applications are performing and proactively identifies issues affecting them and the resources they depend on. It’s an umbrella of services that gives you a way to collect, analyze, and act on all the telemetry from your cloud and on-premises environments. Even though it’s very easy to setup and configure compared to the other services, the visualizations and UI can be improved a lot. I do miss the drill down capabilities from the widgets and graphs added to the dashboard. You can get this working by making use of Power BI or workbooks in the dashboards, but that brings the additional complexities into the service.

Now that we have seen the negatives, I’d like to end this post with the reasons why I’m choosing Azure for my next project as well.


Azure support is great.


They are just fantastic. Fast to respond, always willing to help, trying to find out solutions, providing detailed and updated documentation, free and practical learning paths, hands-on-labs. Contact the support anytime, and you’ll not be disappointed. I’ve noticed that the maximum response time in 2 minutes in Twitter for any queries. That’s simply amazing. Kudos to the spirit and passion @AzureSupport.

Security Center is a complete perfected platform.


Azure Security Center provides unified infrastructure security management that strengthens security posture and provides advanced threat protection across your workloads running in Azure, on-premises, and in other clouds. It is an incredible tool to manage all aspects of security in the Azure Cloud. It’s very easy to create and configure policies, connect to the SIEM solution (Azure Sentinel) and protect your workloads from all kind of attacks and vulnerabilities. You don’t need to be a security expert to effectively configure your infrastructure.
As a CTO/ CIO of a company the tool provides all kind of information and trust that you need from a security solution.

Simple, extensive and up-to-date documentation


The best documentation on any cloud service provider you can find in the internet. Do I need to say more 😊

Thursday, January 31, 2019

Announcing: Hands-On Reactive Microservices in .NET Core Course Published at Packt

I’m very happy to announce that today my new course, Hands-On Reactive Microservices in .NET Core, went live on Packt

This course will take you through the tools and techniques that are necessary to successfully develop, deploy, manage, and monitor microservice-based applications in .NET Core. You'll be developing reactive microservices using .NET Core and Apache Kafka to send messages across microservices. You'll learn about using patterns such as CQRS & Event Sourcing to manage complexities in distributed systems and use Kubernetes to manage deployment, scaling, and updating these services. By the end of the course, you'll be confident in implementing Kubernetes and effectively deploy and manage microservices developed in .NET Core.




Here’s what you'll learn:
  • Use the concepts of reactive programming to build distributed systems running on multiple nodes
  • Develop reactive microservices with .NET Core and Kafka and deploy them on Kubernetes clusters
  • Use docker compose to run and test microservices on a Windows 10 development environment
  • Deploy and manage microservices effectively using Kubernetes tools and resources
  • Monitor microservice infrastructures easily using tools such as ElasticSearch, FluentD, and Kibana

I've followed a hands-on approach that guides your learning through comprehensive examples and clear and concise explanations. You can view the course here. If you don’t have an account yet, don’t worry: you can always start with a free trial.

Thursday, June 7, 2018

My new course : Continuous Delivery with Visual Studio Team Services published


Over the last 3 months I was busy working on my course on Continuous Delivery with Visual Studio Team Services

The course is published by Packt - a leading UK provider of Technology eBooks, Videos and Blogs, and their editors and marketing department has done a fantastic job of helping me with with creative graphics and animations to bring life to this course.


This course is intended for software development professionals who want to learn about Continuous Delivery in a technical value stream as supported by Visual Studio Team Services in order to continually deliver working software at scale. You'll learn how to create, build, test and release a containerized ASP.NET web api to kubernetes cluster on Azure.


The course consists out of seven sections:
  • Introduction to continuous delivery - We'll learn about Continuous Delivery, the business goals and principles of Continuous Delivery
  • Introduction to visual studio team services - We'll learn about the different elements of Visual Studio Team Services and how they function and how using these connected tools you can move your organization to a full-fledged continuous delivery implementation.
  • Setting up the development environment for .net core applications using Docker:
  • Source code management in VSTS : We'll have a look into the Git version control in VSTS and discuss branching strategies like trunk based development and how these concepts relate to continuous delivery.
  • Build management in VSTS : We'll see how to setup a Continuous integration build in VSTS that helps the code stay robust enough that team members can fearlessly make changes to the code base without worrying about breaking existing functionality.
  • Continuous testing : This section we’ll learn how to ensure that an application's components function correctly when assembled together. We'll see how to automate the tests and include them in your Continuous Integration scheme to get the most value from the tests.
  • Release management in VSTS : This section shows you how to do automatic deployments by using Release Management. You'll learn Release Management in VSTS, how it works and how to use release management to deploy our sample application to a Kubernetes cluster on Azure.

The course follows a phased approach to continuous delivery to automate build, deploy and management of .net core applications to Azure Container Services using VSTS ALM. By the end of this course, you'll have the necessary knowledge to create a fully automated build and release pipeline using Visual Studio Team Services for your .NET applications. And don't forget to check the course at https://www.packtpub.com/virtualization-and-cloud/continuous-delivery-visual-studio-team-services-video

Saturday, August 5, 2017

Don't ignore the 'Core' in Continuous Delivery

You see, I'm a person who loves to write code, travel around the world, take nice pictures and eat!!! But nothing less than that, I also enjoy running (especially in the morning!!!, ). It's amazing to have that feeling of the fresh air, smell of the grass and sense the muscles working while observing the surroundings. My running helps me to do my job better because it helps clear my mind, relieves stress and builds a lot of confidence. Its provides me with those moments with great ideas and solutions to most of the problems you have at work or life. By the time you finish running, you have already set up the goal for the day with a certain amount of clarity about the activities that you have to do.

The more I started running, the more I loved challenging myself with those extra miles and pace. But with my overgrowing excitement, I was making the obvious mistake.. I was running too much, too soon, too fast. While doing so, I forgot about the "little things" that help stay healthy and flexible. I started getting injuries, my hips and hamstrings were always tight. I was breaking down with back pain and injuries. I knew I was doing something wrong and it was time to get an expert advice. Having heard the back pain story many times, the physio explained me about the importance of 'Core' muscles and the exercises to do that would strengthen my muscles and improve my core stability. He suggested doing squats, swimming, pull ups, crunches etc. So I decided to give my running a break and focus on core building. It felt a bit off in the gym doing all those non equipment exercises while everyone else was lifting those 20/30 kg dumbbells, but a month after training, I started noticing the difference. I felt more stronger with increased muscle mass, my posture was better, lost some body fat and I was more agile!!! I started running again and this time I noticed the difference immediately. My core helped me to maintain an efficient running form  helping with better stability, balance, posture and overall control.

In short, lesson learnt from my running experience. "Ignoring core strength in your training program may look like working for a short term, but is a recipe for long-term disaster. Lack of core-strength and flexibility causes your body to find a path of resistance towards running or cause the body to adapt in a negative fashion that will most likely lead to complex injuries!!!"

Well how does this relate to continuous delivery and DevOps? The answer is , before immediately adopting a CD tool and start working on the pipeline, you should prepare the teams and the organization with the core of continuous delivery. The core practices like version control, build automation, test automation and continuous integration increases software process agility and helps create an organizational culture and structure to fully embrace CD.

In this post, I’ll describe some of the core practices of continuous delivery  that helps delivering software feature more frequently and reliably. 

Version control everything

Putting everything in version control (source code, build scripts, test scripts and data setup, monitoring scripts etc.) This helps in keeping track and control everything. Keeping track of the changes gives the flexibility of undo or replay any changes easily in all environments including production. You can also understand who made a change and why. Version controlling also creates the first level of built-in safety net, preventing any out of process, locally made changes or manual overrides from entering the production environment. Every change a developer makes must in the source control, any failure to do so will be overridden in the next deployment if not present in the source control.

Practice TDD

TDD is a software development approach where the development of code is driven by first writing an automated test case, followed by writing the code to fulfill the test and then refactoring. With continuous delivery, it’s very important to have the tests pushed into source control along with the implementation code. This helps in ensuring that every new code pushed will be tested in the same build process. This eliminates the risk of delivering to production the code that does not meet quality requirements.

These automated tests helps identify errors in a really short period of time. These tests give developers very fast feedback when something breaks. These tests are also called change detectors. When you make a change, the tests will detect very soon if there are any problems or mistakes. With continuous delivery organizations need the have the ability to make frequent releases of their software without the risk of breaking existing features or adding new bugs. The success of the business depends on how fast it can react to changing environments and address the new improvement suggestions with assurance. Test-driven development is the methodology that makes such flexibility, maintainability, and extensibility possible.

Continuous integration

"Continuous Integration is a software development practice where members of a team integrate their work frequently, usually each person integrates at least daily - leading to multiple integrations per day. Each integration is verified by an automated build (including test) to detect integration errors as quickly as possible. Many teams find that this approach leads to significantly reduced integration problems and allows a team to develop cohesive software more rapidly." - Martin Fowler

Continuous integration process runs ensures that the tests are executed every time a change is pushed into source control and ensures that the new changes didn’t break any other parts of the software. Every push automatically triggers multiple tests. Then if one fails it’s much easier to identify where the error is and start working to fix it.

Automate everything

Automation is the heart of every successful continuous delivery transformation process and the single biggest enabler for CD. The success of continuous delivery lies in stable environments, consistent build and test process and happy releases. Automation can be used to make the release process more deterministic and to bridge the gap between development and production. Development teams can learn from operations the value of automation and how to create automation scripts to improve the overall process. Together the team is responsible for defining the desired state of the infrastructure and convert this to code. The degree to which your organization can achieve successful one click deployment process that can be initiated by anyone whenever needed depends on the level of automation at every stage. At the processes like code analysis, testing, environment provisioning, defect detection and prevention at all stages should be automated. Any manual tasks should be treated as technical debt and later picked up as part of the sprint and addressed.

Test automation

The key to building quality into our software is making sure we can get fast feedback on the impact of changes. Test automation can automate some repetitive but necessary tasks in a formalized testing process already in place, or perform additional testing that would be difficult to do manually. Companies adopting continuous delivery should ensure that when issues are identified they can be managed and resolved quickly and definitively. The easiest and effective way to achieve this is by having an automated test suite incorporated in the release pipeline and CI process. This kind of a process where automated regression, performance and security tests are constantly executed as part of the deployment strategy helps identify issues early and deal with them before they reach the later stages of the deployment cycle/pipeline.

Monitor and measure everything

Proper and effective monitoring of the entire process and environments is important to provide crucial information that ensure service uptime and optimal performance. It's important to measure the progress of the approach to know whether teams are improving or making progress. With the support of proper data and metrics, it’s easy for teams to inspect the current way of working and come up with ideas or processes to improve the overall CD/ DevOps movement. Measuring the current capabilities and process helps teams determine the problem areas and defined focus points for making a change.

An important DevOps practice is to work with application monitoring and insights to enable full end-to-end traceability in a product to provide both operational insights and  usage understanding

Keep focus on these core practices and move forward with your continuous delivery journey!!!
….to satisfy the customer through early and continuous delivery of valuable software” - #1 of the twelve principles behind the agile manifesto.

Sunday, July 16, 2017

Why automated end to end tests may not solve your quality issues

Our customer asked us to help us with an approach for end to end testing. The problem the teams were facing was mainly due to the errors detected during the integration phase/ environment where end to end tests were executed. During the discussions with the teams we noticed that the organizing of teams was based on loosely coupled software components. The teams worked on individual components that were versioned, tested and deployed independently. They had development or test environments were the system tests were executed as part of the build or release process. These system tests were executed against the stubs of the dependent components and verified successfully.

The teams used to get requirements per component which did not specify the entire use case, most of them were in the format of “if a message in format XYZ is given to the component, then do something and send the output to another component in the system in a different format.” Each team developed these user stories and delivered the stories with tests to ensure that the components worked fine, given the correct data is provided in the expected format.

But there was no guarantee that the whole system was working as expected. Most of the time, there were integration issues, due of the wrong versions of components in an integration environments or schema mismatches etc. Different component teams worked in isolation and failed to communicate about the recent releases to other teams. This introduced the need of a separate team to test the system as a whole. The responsibility of this team was to create end to end tests to check and ensure whether the whole system worked properly with multiple components deployed.


The end to end tests now helps in identifying  integration issues before deploying to an higher environment. Only the problem was not completely solved!!!

The issues detected by the integration test team was reported back to the component teams and which was normally not picked up in the current sprint, but has to be put into the backlog for later iterations. This introduced waiting periods and tensions between the different silo teams. Developers were working on parts of the complete applications who were not aware of how the complete system functions were not able to test the system as a whole.

End to end testing was not a solution to this situation, but just a workaround to suppress the problem from getting into higher environments. A better solution will be to organize the teams into feature teams where the team can shift their focus on value delivery and work on business features rather than technical components. Now the teams have acceptance criteria that maps a customer centric use case, rather focusing on a part of the use case. Compared to the component teams, feature teams can provide better design decisions and thus better tests because of the knowledge of the whole application,  technology stack and use cases. The automated tests delivered as part of the user stories now cover the whole feature and not only part of the application.


The feature team as a whole now have the required skills to implement the entire customer centric feature. The integration test team members are also part of the feature teams and assist in ensuring the correct functional tests are captured as part of the development process.

Friday, March 3, 2017

How to create a reliable cluster management system using SQL server for orleans

Orleans runs on a cluster of servers in a data center, each running a container process that creates and hosts actor activation. A service built on Orleans is deployed on a cluster of nodes. When deployed to a cluster of nodes, Orleans internally implements a set of protocols to discover and maintain membership of Orleans silos in the cluster, including detection of node failures and automatic reconfiguration.

Cluster management is supported in Orleans via a built in membership protocol, also referred to as Silo membership. The goal of this protocol is for all silos to agree on the set of currently alive silos, detect failed silos, and allow new silos to join the cluster. For reliable management of cluster membership, Orleans uses Azure Table, SQL Server or Apache ZooKeeper for synchronization of nodes.

This post we’ll see how we can use the SQL server implementation of the membership table and make use of the cluster management feature.

Step 1: Creating the database

Create a new database OrleansMembership in your SQL server and run the below given script to create the required tables and data for SQL server membership provider.



Step 2: Configuring the membership setup in the  SystemStore element settings: 

In the silo configuration file (OrleansConfiguration.xml), add a new section SystemStore in the Globals section with the connection string for the OrleansMembership table we created in step 1.

<OrleansConfiguration xmlns="urn:orleans">
  <Globals>
   
    <Liveness LivenessType="SqlServer" />
    <SystemStore SystemStoreType="SqlServer"
                 DeploymentId="OrleansClusterDemo"
                 DataConnectionString="Data Source=.\SQLEXPRESS;Initial Catalog=OrleansMembership;Integrated Security=True;Pooling=False;Max Pool Size=200;Asynchronous Processing=True;MultipleActiveResultSets=True"
                 AdoInvariant="System.Data.SqlClient" />
   
  </Globals>
</OrleansConfiguration>

Step 3: Update the silo projects with the nuget packages for sql client and sql utils:

"Microsoft.Orleans.OrleansSqlUtils": "1.4.0",
"System.Data.SqlClient": "4.3.0"

Step 4: Update the client configuration with the membership table information: 

The client configuration should also be updated with the OrleansMembership table using the SystemStore element.

<ClientConfiguration xmlns="urn:orleans">
  <Tracing DefaultTraceLevel="Info"
           TraceToConsole="true"
           TraceToFile="C:\OrleansLogs\WebApiClient-{0}-{1}.log"
           WriteTraces="false"/>
  <Statistics MetricsTableWriteInterval="300s" PerfCounterWriteInterval="30s" LogWriteInterval="300s" WriteLogStatisticsToTable="true" StatisticsCollectionLevel="Info"/>
  <Messaging ResponseTimeout="30s" ClientSenderBuckets="8192" MaxResendCount="0"/>
  <SystemStore SystemStoreType ="SqlServer"
                 DeploymentId="OrleansClusterDemo"
                 DataConnectionString="Data Source=.\SQLEXPRESS;Initial Catalog=OrleansMembership;Integrated Security=True;Pooling=False;Max Pool Size=200;Asynchronous Processing=True;MultipleActiveResultSets=True" AdoInvariant="System.Data.SqlClient" />
</ClientConfiguration>

Also notice that you don't need to use the ProxyingGateway anymore for the Primary silo address. Instead the client can now use the membership data for grain references.



Conclusion:

We have created a reliable deployment that is configured with membership using SQL server. In this case, all silos are created equal, with no notion of a primary of secondary silos. This is the recommended production deployment scenario where a failure of a single node or a combination of nodes does not affect other silos in the cluster. This also means that you don't need to provide a Seed node option in your silo configuration anymore!!!

Sunday, February 26, 2017

How to create a monitoring infrastructure for your Orleans silos and grains

Orleans dashboard is a configuration based bootstrap provider for Orleans silos. The dashboard provides details on the performance of silos, individual grains and also the details on failures etc.

If you are new to actor systems and orleans, it may be helpful to read the official documentation https://dotnet.github.io/orleans/index.html



How do you use it?

First thing you want to do is install the OrleansDashboard packages via NuGet to the GrainServer project (this is where the gains are implemented)

Install-Package OrleansDashboard

Register the dashboard bootstrapper into the Orleans system

In the OrleansConfiguration.xml file add the new bootstrapper. You can mention the port number on which the dashboard will be accessible. The default value is 8080.
<OrleansConfiguration xmlns="urn:orleans">
  <Globals>
    <SeedNode Address="localhost" Port="11111" />
    <StorageProviders>
      <Provider Type="Orleans.StorageProviders.SimpleSQLServerStorage.SimpleSQLServerStorage" Name="SqlStore"
                ConnectionString="Data Source=.\SQLEXPRESS;Database=OrleansStore; Integrated Security=True;"
                UseJsonFormat="false" />
    </StorageProviders>
    <BootstrapProviders>
      <Provider Type="InfraAutomation.GrainServer.BootstrapServer" Name="BootstrapServer" />
      <Provider Type="OrleansDashboard.Dashboard" Name="Dashboard" Port="8081" />
    </BootstrapProviders>
  </Globals>
....


Start the silo and you can now access the dashboard at the configured port like http://localhost:8081