Using the combination of TFS API’s and PowerShell, you can
easily manage your TFS test cases, either as part of a build process or from a
tool that supports invoking PowerShell cmdlets or scripts. In this post, I’ll
show how to make use of the [Microsoft.TeamFoundation.TestManagement.Client]
API’s to manage test runs in TFS or Microsoft Test Manager. The test management
client API’s are internally used by MS visual studio Ultimate/ Test
Professional editions to manage TFS test plans, and cases.
At the end we’ll create a PowerShell module with the
functions that are needed for managing the test runs in TFS and use the functions
to meet our requirements. The rest of the post contains sections defining the
functions that are exposed as part of the Module.
Assemblies involved in the example:
The first step is to load the assemblies which are needed for
our functions to interact with the TFS API’s
[void][System.Reflection.Assembly]::LoadWithPartialName("Microsoft.TeamFoundation.Client")
[void][System.Reflection.Assembly]::LoadWithPartialName("Microsoft.TeamFoundation.TestManagement.Client")
[void][System.Reflection.Assembly]::LoadWithPartialName("Microsoft.TeamFoundation.Build.Client")
The Microsoft.TeamFoundation.Build.Client assembly is used to
get the details of the latest build that will be used to create a test run.
Get-TeamProjectCollection
In TFS a
team project collection is used to group multiple team projects that needs a
common set of resources or have a similar set of requirements or objectives. A team project collection can be retrieved by
using the GetTeamProjectCollection method on the
TfsTeamProjectCollectionFactory class.
Function Get-TeamProjectCollection
{
[CmdletBinding()]
param
(
[Parameter()]
[ValidateNotNull()]
[Uri]$Uri
)
if(-not($Uri))
{
$Uri
= "http://yourtfsserver:8080/tfs/defaultcollection"
}
[Microsoft.TeamFoundation.Client.TfsTeamProjectCollectionFactory]::GetTeamProjectCollection($Uri)
}
PS C:\>
Get-TeamProjectCollection
Name : tfsprajeesh\DefaultCollection
DisplayName : tfsprajeesh
CatalogNode : CatalogNode instance 59865478
ChangeTypeValue: 0
IsDefault: False
MatchedQuery: False
NodeDependencies: [0]
NodeDependenciesIncluded: False
Get-TeamProject
A team
project is a collection of work items, code, tests, work products, metrics, and
so forth that is used by a defined team to track a common set of related work.
The logical concept of a team project helps you determine what to include or
exclude from the development of a software application.
In our
sample, we will make use of the TestManagement API (you can also use the TeamFoundation.Client
API to get the team project) to get the TeamProject object.
Function Get-TeamProject
{
[CmdletBinding()]
param
(
[Parameter(ValueFromPipeline=$true, Position=1)]
[ValidateNotNull()]
[Microsoft.TeamFoundation.Client.TfsTeamProjectCollection]$TeamProjectCollection,
[Parameter(Mandatory=$true, Position=0)]
[ValidateNotNullOrEmpty()]
[String]$Name
)
$service =
$TeamProjectCollection.GetService([Microsoft.TeamFoundation.TestManagement.Client.TestManagementService])
$service.GetTeamProject($Name)
}
PS C:\>
Get-TeamProjectCollection | Get-TeamProject -Name "TeamProject1"
TestEnvironments :
Microsoft.TeamFoundation.TestManagement.Client.TestEnvironmentHelper
IsValid : True
DefaultPageSize : 200
DefaultPageSizeForTestPoints
: 400
Get-TestPlan
A test plan
lets you specify what you want to test and how to run those tests. You can
create Test plans in Microsoft Test Manager or using the TFS API's. For more
details on Test plans and how to create them follow the doumentation at https://msdn.microsoft.com/en-us/library/vstudio/dd286583(v=vs.110).aspx
. To get a test plan instance based on the test plan name or test plan id,
we'll make use of the TestPlans collection in the TeamProject object as given
below.
Function Get-TestPlan
{
[CmdletBinding()]
param
(
[Parameter(Mandatory=$true, ValueFromPipeline=$true,
Position=0)]
[ValidateNotNull()]
$TeamProject,
[Parameter(Position=1)]
$Id,
[Parameter(Position=2)]
$Name
)
if($Id)
{
$TeamProject.TestPlans.Find($Id)
}
else
{
$TeamProject.TestPlans.Query("SELECT * FROM TestPlan WHERE PlanName='$Name'")
}
}
PS C:\>
Get-TeamProjectCollection | Get-TeamProject -Name "TeamProject1" |
Get-TestPlan -Name "TestPlan1"
ObjectType : TestPlan
Name : TestPlan1
Description :
Get-TestSuite
By using
Microsoft Test Manager, you can organize test cases into a hierarchy of test
suites inside test plans. For more information on test suites, follow the
documentation at https://msdn.microsoft.com/en-us/library/vstudio/dd286738(v=vs.110).aspx.
We can make use of the TestSuites collection of the TeamProject object to query
a test suite based on the Id.
Function Get-TestSuite
{
[CmdletBinding()]
param
(
[Parameter(Mandatory=$true, Position=0,
ParameterSetName="TeamProject")]
[ValidateNotNull()]
$TeamProject,
[Parameter(Mandatory=$true,
ParameterSetName="TeamProject")]
[Parameter(Mandatory=$false,
ParameterSetName="TestPlan")]
[ValidateNotNull()]
$Id,
[Parameter(Mandatory=$true, ValueFromPipeline=$true,
ParameterSetName="TestPlan")]
[ValidateNotNull()]
[Microsoft.TeamFoundation.TestManagement.Client.ITestPlan]$TestPlan
)
switch($PsCmdlet.ParameterSetName)
{
"TeamProject"
{
$TeamProject.TestSuites.Find($Id)
break;
}
"TestPlan"
{
$TestPlan.Project.TestSuites.Find($Id)
break;
}
}
}
PS C:\>
Get-TeamProjectCollection | Get-TeamProject -Name "TeamProject1" |
Get-TestPlan -Name "TestPlan1" | Get-TestSuite -Id 2
TestSuiteType : StaticTestSuite
ObjectType : TestSuite
Plan : TestPlan instance 23
Get-TestEnvironment
A lab
environment is a group of computers that you manage as a single entity. Usually
you use them for system testing. If you’re testing a distributed application
such as a web app, you can perform realistic tests by deploying each component
on a separate machine.
Lab
environments let you collect diagnostic data from all the machines in the lab
while you’re performing your tests. The data, such as event logs or
Intellitrace files, are attached to the test results and to any bug that you
create. For more details on Lab environments refer to the documentation https://msdn.microsoft.com/en-us/library/ee943321.aspx.
The TeamProject has a collection of TestEnvironments which we can Query the
Environment by name or id
Function Get-TestEnvironment
{
[CmdletBinding()]
param (
[Parameter(Mandatory=$true, ValueFromPipeline=$true,
Position=0)]
[ValidateNotNull()]
$TeamProject,
[Parameter(Position=1)]
$Name
)
$TeamProject.TestEnvironments.Query() | ? { $_.Name.EndsWith($Name) }
}
PS C:\>
Get-TeamProjectCollection | Get-TeamProject -Name "TeamProject1" |
Get-TestEnvironment -Name "Env1"
Id : bgvf6fb7-96f3-483c-4356-078148cf45de
Name : TeamProject1.0.Env1
Description :
Get-TestConfiguration
Test configurations
will help users to test applications with different configurations like web
browsers, different operating systems etc. A detailed description about test
configurations is provided here https://msdn.microsoft.com/en-us/library/vstudio/dd286643(v=vs.110).aspx.
The TestConfigurations collection property in the TeamProject can be used to
find and use a specific configuration during a test run.
Function Get-TestConfiguration
{
[CmdletBinding()]
param
(
[Parameter(Mandatory=$true, ValueFromPipeline =
$true)]
[ValidateNotNull()]
$TeamProject,
[Parameter(Mandatory=$false,
ParameterSetName="Id",
Position = 1)]
[int]$Id,
[Parameter(Mandatory=$false,
ParameterSetName="Name",
Position = 1)]
[string]$Name
)
switch($PsCmdlet.ParameterSetName)
{
"Id"
{
$TeamProject.TestConfigurations.Find($Id)
break;
}
"Name"
{
$TeamProject.TestConfigurations.Query("SELECT * FROM TestConfiguration WHERE Name='$Name'")
break;
}
}
}
PS C:\>
Get-TeamProjectCollection | Get-TeamProject -Name "TeamProject1" |
Get-TestConfiguration -Name "Internet Explorer Browser"
ObjectType : TestConfiguration
Name : Internet Explorer Browser
Description : Use this configuration to run an
automation test in internet explorer
AreaPath : TeamProject1
IsDefault : False
Get-TfsBuild
During a test run, it’s important to specify the TFS build to
be used that contains the test methods that the tests are executed against. A
team build can be queried using the Microsoft.TeamFoundation.Build.Client API,
by providing the TeamProject and Build definition name.
Function Get-TfsBuild
{
[CmdletBinding()]
param
(
[ValidateNotNull()]
[Microsoft.TeamFoundation.Client.TfsTeamProjectCollection]
$TeamProjectCollection,
[Parameter(Mandatory=$true)]
[ValidateNotNullOrEmpty()]
[string]
$TeamProject,
[Parameter(Mandatory=$true)]
[ValidateNotNullOrEmpty()]
[string]
$BuildDefinition,
[string]
$BuildNumber
)
$server = $TeamProjectCollection.GetService([Microsoft.TeamFoundation.Build.Client.IBuildServer])
if($BuildNumber)
{
$server.QueryBuilds($TeamProject,
$BuildDefinition) |? {$_.BuildNumber -eq
$BuildNumber}
}
else
{
$server.QueryBuilds($TeamProject,
$BuildDefinition) |? {$_.Status -eq "Succeeded"} |
Select -Last 1
}
}
PS C:\> Get-TfsBuild
-TeamProjectCollection $collection -TeamProject $teamProject.TeamProjectName
-BuildDefinition BlogsPrajeesh.Samples.Web_CI | select BuildNumber
BuildNumber
-----------
1.0.0.5
Invoke-TestRun
A test run
is the execution of test cases in a test suite or test case, to be executed
against a specific configuration in an test environment. You also need to
mention a TFS build to be used for the test run. For details on how to run
tests in Microsoft test manager follow the article https://msdn.microsoft.com/en-us/library/dd286680(v=vs.110).aspx.
In our
function, we will make use of the output from the previous functions created in
this post to start a test run. Later we’ll create functions to get the result
of a test run.
Function Invoke-TestRun
{
[CmdletBinding()]
param(
[Parameter(Mandatory=$true)]
[ValidateNotNull()]
$TeamProject,
[Parameter(Mandatory=$true)]
[ValidateNotNull()]
[string]$Title,
[Parameter(Mandatory=$true)]
[ValidateNotNull()]
$TestPlan,
[Parameter(Mandatory=$true)]
[ValidateNotNull()]
$TestSuite,
[Parameter(Mandatory=$true)]
[ValidateNotNull()]
$TestConfiguration,
[Parameter(Mandatory=$true)]
[ValidateNotNull()]
$TestEnvironment,
[Parameter(Mandatory=$true)]
[ValidateNotNull()]
[Microsoft.TeamFoundation.Build.Client.IBuildDetail]$Build
)
$TestRun = $TestPlan.CreateTestRun($true)
$TestRun.BuildUri
= $Build.Uri
$TestRun.BuildNumber
= $Build.BuildNumber
$TestRun.BuildDirectory
= $Build.DropLocation
$QueryText =
"SELECT * FROM TestPoint WHERE SuiteId=$($TestSuite.Id) AND ConfigurationId=$($TestConfiguration.Id)"
$TestPoints =
$TestPlan.QueryTestPoints($QueryText)
$TestPoints |
? { -not($_.State -eq "NotReady"
-and $_.MostRecentResultOutcome -eq
"Blocked") -and $_.MostRecentResultOutcome -ne
"NotApplicable" } | % { $TestRun.AddTestPoint($_, $null) }
$TestRun.Title
= $Title
$TestRun.Owner
= $TestTeamProject.TestManagementService.AuthorizedIdentity
$TestRun.TestEnvironmentId
= $TestEnvironment.Id
$TestRun.Controller
= $TestEnvironment.ControllerName
$TestRun.Save()
$TestRun
}
PS C:\>
Invoke-TestRun -TeamProject $teamProject -Title "Demo Testrun2" -TestPlan
$plan -TestSuite $suite -TestConfiguration $config -TestEnvironment
$environment -Build $build
ObjectType : TestRun
BuildDirectory : \\TFSSER\TRUNK\1.0.0.5
State : Waiting
PostProcessState : Complete
DateDue : 1-1-0001 0:00:00
Statistics :
Microsoft.TeamFoundation.TestManagement.Client.TestRunStatistics
ErrorMessage :
Iteration : TeamProject1
LegacySharePath :
Type : Unspecified
IsAutomated : True
Version : 1000
IsBvt : False
TotalTests : 0
IncompleteTests : 0
NotApplicableTests : 0
PassedTests : 0
UnanalyzedTests : 0
TestMessageLogEntries :
{}
Title : Demo Testrun2
BuildUri : vstfs:///Build/Build/5
BuildNumber : 1.0.05
BuildConfigurationId : 0
BuildPlatform :
BuildFlavor :
DateCreated : 1-1-0001 0:00:00
DateStarted : 1-1-0001 0:00:00
DateCompleted : 1-1-0001 0:00:00
LastUpdated : 4-7-2015 9:33:00
LastUpdatedBy : TeamFoundationIdentity instance 34352
Receive-TestRun
The receive
test run method gets the results of a test run at a later point of time by
providing a TestRun Id.
Function Receive-TestRun
{
[CmdletBinding()]
param(
[Parameter(Mandatory=$true, Position=0,
ParameterSetName="TeamProject")]
[ValidateNotNull()]
$TeamProject,
[Parameter(Mandatory=$false, ParameterSetName="TestPlan")]
[Parameter(
ParameterSetName="TeamProject")]
[int]$Id,
[Parameter(Mandatory=$true, ValueFromPipeline=$true,
ParameterSetName="TestPlan")]
$TestPlan
)
switch($PsCmdlet.ParameterSetName)
{
"TeamProject"
{
$TeamProject.TestRuns.Find($Id)
break;
}
"TestPlan"
{
$TestPlan.Project.TestRuns.Query("SELECT *
FROM TestRun WHERE TestPlanId=$($TestPlan.Id) AND TestRunId=$Id")
break;
}
}
}
Wait-TestRun
The wait
run is invoked in conjunction with the Invoke-TestRun, to wait till the test
run is completed. This is useful when you are using the Invoke-TestRun method
as part of a Tfs Build activity.
Function Wait-TestRun
{
[CmdletBinding()]
param
(
[Parameter(Mandatory=$true, ValueFromPipeline=$true)]
[ValidateNotNull()]
[Microsoft.TeamFoundation.TestManagement.Client.ITestRun]$TestRun
)
do
{
Start-Sleep -Seconds 5
$TestRun = Receive-TestRun -Id $TestRun.Id -TeamProject $TestRun.Project
}while($TestRun.State -ne "Completed" -and
$TestRun.State
-ne "NeedsInvestigation"
-and $TestRun.State -ne "Aborted")
$TestRun
}
$run = Invoke-TestRun
-TeamProject $teamProject -Title "Demo Testrun2" -TestPlan $plan
-TestSuite $suite -TestConfiguration $config -TestEnvironment $environment
-Build $build | Wait-TestRun
