Bringing Agile into SharePoint - ALM

SharePoint development platform, provides the capabilities to develop, deploy, and update customizations and custom functionalities for the SharePoint sites. The activities that take advantage of these capabilities all fall under the category of Application Lifecycle Management (ALM).

“ALM is a continuous process of managing the life of an application through governance, development and maintenance.”

With the options like BCS, Client OM, Sandbox solutions, WCF based Service Applications and Claims based authentication SharePoint has also significantly evolved as application development platform which opens up a number of extensibility options for the developers. With the power and flexibility to use these tools and customizations the question arises is how to effectively and efficiently manage the activities taking into account the lifecycle of application starting from requirement gathering to release management with consistent quality.

The entire process involves setting up SharePoint with visual studio, TFS, Build automation, Deployment and automated testing.

Setting up DEV environment

To take the full advantage of development capabilities, SharePoint is installed on all development machines. We use scripted installation of SharePoint (SPAutoInsaller) which helps to maintain a similar environment for all development machines and helps the team to configure SharePoint easily.

TFS as a source control

With Visual studio as a development tool, the integration to source control is much easier as the IDE is easily integrated to the TFS. The team creates and works on team projects, checks in the code directly from Visual Studio to TFS.

TFS for build automation

CI builds are configured to ensure that every code that is checked in passes the build with the configured build standards. On every code check-in the build server to pull down the latest source code, compile the code, run any static analysis rules, executes all unit tests and integration tests on the latest code and report any errors along the way. This is a great way of ensuring that even if the developers locally don’t run the static analysis rules or unit tests they will be run here and also ensures that all source code is checked in and is working as expected.

 

 

TFS for Code analysis

Code analysis rules help teams to ensure that coding is done according to the standards defined and are followed without fail. The default set of rules are built into Visual Studio which can also be tweaked to meet the specific needs of the teams. Apart from the built in rules which are common to all projects, additional rules like SPDisposeCheck that analyzes the code to see whether all SPWeb and SPSite objects are disposed correctly can be added to the rules. These rules are run on both Visual Studio and TFS to ensure continuous quality.

Automated deployment

Automated deployment is triggered as part of the build process, or as an unattended service to deploy solution packages. The deployment process is a combination of batch files, PowerShell commands and xml configuration files. It is a great way to update the test machines with latest version of the software every time a new version is checked in.

Automated unit, smoke and regression testing

Testing is also configured as part of the CI build process or a nightly build process depending on the teams requirements. Most of the unit tests are executed on every check-in build and regression tests are executed as part of the nightly builds. Team focuses on creating and maintaining a clean set of test cases that help the team get an early feedback on the quality of the product after every release to the test environment.

 

Bringing Agile into SharePoint - CI and DTAP

Creating and maintaining a DTAP (Development-Test-Acceptance-Production) environment helps the project teams and IT department ensure faster and quality delivery at a lower cost. It also reduces the risk of breaking other applications when side by side deployment of components is facilitated in higher environments. The teams get early feedback on the compatibility and performance of the software in Production by simulating the deployment on a production like environment (A). All stakeholders (including the customer) can be involved in the acceptance of the feature before the feature is deployed in Production by following the DTAP street.

The code checked in by the team is rebuild on a daily basis with unit test and deployed daily to a test environment (CI box). All changes to CI and subsequent environments are deployed using wsp's and PowerShell scripts. A fully automated CI process is implemented by a combination of XML configuration files and PowerShell scripts. Having a process like this will also ensure that installation on all environments including development have a similar process.

Environments and purpose

Development (D)

Standalone VM's with self-contained SQL server, SharePoint, Visual Studio and related tools for developers to create and debug solutions.  Developers work on code checked into TFS and are responsible for maintaining a passing build on every check-in. These servers are also used as quick demo environments to end users and building proof of concepts.

The development machines are configured by using PowerShell scripts which makes it easy to configure SharePoint on machines by team members who have minimal knowledge of SharePoint.

Test (T)

This environment is mainly used by IT department. Teams make use of this environment to fine tune installations before moving up to the next farm in the street. This environment has a stable release so testers can perform better integration testing against a specific release build. Project teams use this environment for demos to end users and have installations multiple times before going to A.

Acceptance (A)

This a production like server where the IT department performs final validation of the software before going to production. This environment is not used for final testing, but to use as a stage to get final approval from the PO before going to production.

Production (P)

This is the final phase of the development process. The installations are all fully automated and have the content to serve the end users!!!

Bringing Agile into SharePoint - Quality check

One of the major problems with SharePoint development is that the framework is not designed with testability in mind. It is not easy to replace the SharePoint object model using Mocks/ Stubs which makes practices like unit testing more difficult in SharePoint development.

The main focus will be to use a combination of best architectural practices and isolation frameworks to achieve best results to do Agile in SharePoint. For e.g. isolating SharePoint calls using a Repository pattern makes it easy to mock a repository object in the above layers to isolate SharePoint object model. Building SharePoint solutions using XP techniques helps the teams in faster development, predictable deliveries, better quality, less stress and happy stakeholders

Repositories

Repositories act as mediators, or interface layers, between different parts of an application. Making use of the repository pattern the SharePoint object model can be exposed using a and repository which will be responsible for using the SharePoint API for querying and updating SharePoint list items.

By using a combination of Specification and data mapper pattern with Repositories will allow the team to centralize the SharePoint list access logic and conversion of types. This provides a simplified access mechanism of list data and provide a layer of abstraction that makes unit testing with emulators/mocks much easier

 

MVP

Separating the logic for the UI from the code that handles user events, fetching data from repositories by using an implementation of MVP allows developers to replace the actual view with a mock object for unit tests.

SharePoint Emulator

With the availability of SharePoint Emulators the isolation of the SharePoint API is much easier compared to the old way of isolating using the Moles or Fakes framework.

In order to incorporate emulators into existing tests you should wrap the relevant code in a SharePointEmulationScope, which accepts EmulationMode enum as a parameter. It is enabled by default which in turn performs run time interception of all Microsoft.SharePoint.dll calls.

[TestMethod]

[TestCategory("Data access layer tests")]

[Description("Test to verify the data mapper functionality")]

public void BuildFromSPListItem_should_create_a_new_object_with_populated_values()

{

    using (new SharePointEmulationScope(EmulationMode.Enabled))

    {

        var dataMapper = new MyObjectDataMapper();

        const string customDescription = "Custom Description";

        using (var emulatedSite = new SPSite("http://localhost:8081/MySite"))

        {

            var listId = emulatedSite.RootWeb.Lists.Add("MyList", "Custom list", SPListTemplateType.GenericList);

            var list = emulatedSite.RootWeb.Lists[listId];

            var listitem = list.Items.Add();

            listitem["Title"] = "mytitle";

            listitem["CustomDesc"] = "Custom Description";

            var myObject = dataMapper.FromSPListItem(listitem);

            Assert.IsTrue(myObject.CustomDesc == customDescription);

        }

    }

}

Smoke tests

Smoke tests are quick, lightweight test or set of automated tests, that confirm that the basic functionality of the sites are working as expected. We use the smoke tests to verify whether the basic standards of internet sites are adapted and are working as expected in all the environments. The smoke tests are executed from a set of configuration files that specify the details of the sites and environments to run against.

By having proper smoke tests, the team can ensure that the site works with the basic functionalities after every upgrade or patches are done on the farm.

Automation tests

The team also creates a set of UI driven automated tests that cover the functional testing of the UI controls. This helps us verify that the whole application including the user interface is functioning correctly. Teams create Coded UI tests that help remove the need for manual regression testing of a functionality. The Coded UI tests simulates the user experience of clicking through the UI controls to verify things are working correctly.

Bringing Agile into SharePoint - Benefits

Learn early

Doing sprints help the team identify issues early at the project life cycle. This helps the team find things that are good or bad for the team or process early in the development stage rather than waiting till the end of the project to get feedback. This way the teams Learn early from the mistakes and can better themselves in the upcoming sprints.

Potentially shippable software at the end of every sprint.

The intention of every sprint is a potentially shippable software that is of immediate use to the user. By delivering a product that can be used and tested will also help the team receive early feedback on the work done. The team can then make changes to the product based on these feedback which will eventually result in a more value delivered to the end users.

This will also help the team perceive shippable obstacles and issues early in the development state compared to the waterfall model where issues related to release of software is often identified at the end.

Kaizen mindset

The scrum artifacts like daily standups, sprint reviews, retrospections help the team to focus on continuous improvement through inspect and adapt. The team manges to improve by identifying and managing the impediments during the process.

Better visibility and improved productivity

The team focuses on keeping the progress of the project visible to the entire world. Everything the team is busy with is displayed on the tailboard. The task board shows the progress of the stories or tasks, spikes, bugs and the status of each item.

Burn down charts facilitate as symptom checkers for the team and help them to identify the smells during the progress of the sprint. For e.g a flat burndown shows that either the team is working on a lot of stories at time or nothing is getting done completely  in the sprint.

The velocity charts helps business envision and evaluate the productivity of the team after every sprint.

Bringing Agile into SharePoint - The combination

SharePoint offers several flexible approaches that enable rapid development and deployment of business processes through its out-of-the-box features which are also easily customizable. A SharePoint developer can leverage a lot of this out of box functionality and combine it with .NET skills to create highly functional business solutions quickly, easily and at a fraction of the cost of traditional solutions.

A combination of SharePoint and Agile can be used to deliver good solution to the customers with improved speed up overall delivery time with better focus on developer productivity.

So what are the benefits that SharePoint lends to Agile?

A lot of out of the box functionality:
Most of the client requirements map directly onto features provided from a default provision of SharePoint. With a wide choice of from web templates to workflows most of the features SharePoint has supports the vision for the customers’ requirements.

Rapid prototyping:
SharePoint comes with an abundance of features and tools that help developers quickly build business solutions even if they don’t have a strong background in web development. You can easily build, customize and deploy solutions using SharePoint. It’s very simple to create a rough working model of what the customer is looking for that helps the team to ensure that they can deliver what the customer really wants using SharePoint.

Easy customization:
With SharePoint, you have an option of OOTB features with which the development team has the ability to build custom applications and components with ease. Along with the help of proper tooling the administrators can customize the entire SharePoint experience with your his/ her organizations branding.

Easy deployment and administration:
Administrators can easily access and manage features, system settings, monitor farms, perform backup and restore content, security through the Central Administration Console all in a single location.

I'll write more on the combination of Agile and SharePoint in the upcoming posts and we'll also see how to address some of the challenges of Agile development practices in SharePoint like unit testing, TDD etc.

Unit test generator extension - Visual studio 2012, 2013

The new unit test generator extension RC1 to visual studio was introduced by ALM Rangers that helps developers add "Create unit test" feature to visual studio. The new feature supports multiple framework targets, automated test project creation and configuration.

The feature comes with a configuration menu, which allows you mention the Test framework, naming conventions and options to construct the test method body on creation of unit tests.

You can now simply right click on a class with public methods and select Create unit tests to see the feature in action.

After test generation you can see the test class with the respective test methods added as given below.

using Microsoft.VisualStudio.TestTools.UnitTesting;

namespace CQRS.LibTests

{

    [TestClass()]

    public class RespondToRsvpCommandValidatorTests

    {

        [TestMethod()]

        public void ValidateTest()

        {

            Assert.Fail();

        }

    }

}

Distributing custom stylecop rules in a scrum team

The Microsoft source analyzer for C#, StyleCop can be used to enforce a set of styling and consistency rules among .Net teams in a project/ company. StyleCop can be run as a visual studio plugin or can be integrated with an MSBuild project.

StyleCop provides an extensible framework for plugging in custom rules for the developers.  For implementing a custom rule, the user needs to create a custom rules analyzer class which inherits the SourceAnalyzer and overrides the AnalyzeDocument method to check for violations.

In this post, I’ll show how you can implement extensible custom rules in stylecop by using the visitor pattern.

Creating the custom rule analyzer class:

[SourceAnalyzer(typeof(CsParser))]

public class MyCodeStandardAnalyzerRules : SourceAnalyzer

{

    public override void AnalyzeDocument(CodeDocument document)

    {

        var csDoc = document as CsDocument;

        if(csDoc == null) return;

        if(csDoc.HasEmptyRootElement() || csDoc.IsAutoGeneratedCode()) return;

        csDoc.WalkDocument(

                ElementVisitor, null, null

            );

    }

    private bool ElementVisitor(CsElement element, CsElement parentelement, object context)

    {

        //Implementing the custom rules for CsElement goes here

    }

}

Creating the visitor interface

public interface IVisitor

{

    void SetSourceAnalyzer(SourceAnalyzer alanyzer);

    void Visit(CsElement element);

}

Writing the first visitor

public class CheckForConstantsHaveUpperCaseNameVisitor : BaseVisitor, IVisitor

{

    public void Visit(CsElement element)

    {

        if (element.ElementType != ElementType.Field || !element.Declaration.ContainsModifier(CsTokenType.Const))

            return;

        if(element.Name.Any(char.IsLower))

            AddViolation(element, RuleNames.CONSTANTS_SHOULD_BE_IN_UPPERCASE, element.Name, element.LineNumber);

    }

}

public class BaseVisitor

{

    private SourceAnalyzer _sourceAnalyzer;

    public void SetSourceAnalyzer(SourceAnalyzer sourceAnalyzer)

    {

        if (sourceAnalyzer == null) throw new ArgumentNullException("sourceAnalyzer");

        _sourceAnalyzer = sourceAnalyzer;

    }

    protected void AddViolation(ICodeElement element, string rule, params object[] values)

    {

        _sourceAnalyzer.AddViolation(element, rule, values);

    }

}

Creating the Element class to accept the visitor

public class CodeElement

{

    private readonly SourceAnalyzer _analyzer;

    private readonly CsElement _element;

    public CodeElement(SourceAnalyzer analyzer, CsElement element)

    {

        _analyzer = analyzer;

        _element = element;

    }

    public void Accept(IVisitor visitor)

    {

        visitor.SetSourceAnalyzer(_analyzer);

        visitor.Visit(_element);

    }

}

The visitor dispatcher to resolve all visitors (Using a dependency injection container is much easier J)

public class VisitorDispatcher

{

    private static VisitorDispatcher _dispatcher;

    private static readonly object _lockObject = new object();

    public List<IVisitor> Visitors { get; private set; }

    public static VisitorDispatcher Instance

    {

        get

        {

            if (_dispatcher == null)

            {

                lock (_lockObject)

                {

                    _dispatcher = new VisitorDispatcher();

                    var visitors = from type in Assembly.GetExecutingAssembly().GetTypes()

                                    where type.IsAssignableFrom(typeof(IVisitor))

                                    select Activator.CreateInstance(type) as IVisitor;

                    _dispatcher.Visitors = new List<IVisitor>(visitors);

                }

            }

            return _dispatcher;

        }

    }

}

Finally visiting the element with the visitors

private bool ElementVisitor(CsElement element, CsElement parentelement, object context)

{

    if (element.IsAutoGenerated()) return true;

    var codeElement = new CodeElement(this, element);

    VisitorDispatcher.Instance.Visitors.ForEach(codeElement.Accept);

    return true;

}

The last step is to build your project and drop the new project’s dll into the StyleCop installation directory and run the style cop rules on your projects. Style cop will automatically look for assemblies in the directory and pick up the new rules for your team.