Category: TFS

Override the TFS Team Build OutDir property in .NET 4.5

Update: with Team Build 2013 it is easier still.

I’ve blogged before about the challenge of overriding the OutDir MSBuild property set by Team Build but this hassle is gone in version 4.5 of the .NET Framework.

I stumbled upon a change to the core Microsoft.Common.targets file while trying to understand some build issues with a work project and discovered new logic to modify the OutDir property depending on a variety of conditions. I went searching through the rest of the file for other references to OutDir and also discovered at the top of the file, a new attribute on the Project element. This new attribute’s name is “TreatAsLocalProperty” and it’s value is simply “OutDir”.

As at the time of posting this blog entry I could not find any documentation of this new functionality but based on my own testing I found that .NET Framework 4.5 now supports:

  • Overriding the value of an MSBuild property that was specified at the MSBuild command-line by naming that property in the TreatAsLocalProperty attribute at the top of the build project.
  • OutDir can now be specified at the command-line without a trailing slash and it will be corrected for you instead of failing the build.
  • Projects can automatically build to subfolders of the Team Build drop location by setting a new MSBuild property named “GenerateProjectSpecificOutputFolder” to “true”.
  • The project-subfolder will be named the same as the project file but can be overridden by specifying an alternate value for the “ProjectName” MSBuild property.
  • The OutDir property can now be overridden in whatever custom way you like without modification of the Team Build workflow xaml or using a before-solution targets file.

And because .NET 4.5 is an addition to .NET 4 in the same way .NET 3.5 and .NET 3.0 were to .NET 2.0, your existing .NET 4/VS2010 projects can benefit from this new build-time functionality without taking on new run-time dependencies (with some exceptions). Here is a screenshot for how to configure Team Build 11 or a Team Build 2010 server with .NET 4.5 installed to create per-project folders in the build drop:

TFS Lab Build needs to start the environment

Most of the Team Foundation Server virtual lab environments I work with use snapshots that are taken while the lab virtual machines are running. When configuring a Build Definition with the default lab build process template (LabDefaultTemplate.xaml) and choosing to restore the environment to one of these snapshots everything just works. However, the guidance from Microsoft around taking snapshots of certain virtual machines (primarily Active Directory domain controllers and occasionally SQL Servers) is to shutdown the VMs first before taking the snapshot.

In recent work with a lab containing a domain controller, we’ve followed the recommendation of performing the snapshot while the VM is off. As a result of this I’ve discovered that the default lab build process template will restore to the selected snapshot and wait 20 minutes for the workflow capability of the environment to become ready and then fail. It never checks if the environment is started and never attempts to start it.

I’ve since added a call to the GetLabEnvironmentStatus and StartLabEnvironment activities into the lab build process template just after the “Restore Snapshot” sequence and before the “Do deployment” sequence so the environment gets started if it isn’t already running.

The full customised build process template is available as a Gist. The changes are small though – first a new variable definition is inserted at line 30:

<Variable x:TypeArguments="mtlc:LabEnvironmentState" Name="LabEnvironmentState" />

And then 6 lines are inserted at line 81 (was line 80 before the above line was added):

<mtlwa:GetLabEnvironmentStatus LabEnvironmentUri="[LabEnvironmentUri]" Result="[LabEnvironmentState]" DisplayName="Get Lab Environment State" />
<If Condition="[LabEnvironmentState &lt;&gt; Microsoft.TeamFoundation.Lab.Client.LabEnvironmentState.Running]" DisplayName="If Lab Environment Not Running">
  <If.Then>
    <mtlwa:StartLabEnvironment LabEnvironmentUri="[LabEnvironmentUri]" DisplayName="Start Lab Environment"/>
  </If.Then>
</If>

A very similar change will be required in TFS 11 because its lab build process still won’t start an environment. Apparently this is to avoid making assumptions about the order in which each VM in a lab should be started.

Test Attachment Cleaner Least Privilege

I manage a Team Foundation Server 2010 instance with at least 30 Collections each with several Team Projects. Even after installing the TFS hotfix to reduce the size of test data in the TFS databases we still accrue many binary files care of running Code Coverage analysis during our continuous integration builds. As such it is still necessary to run the Test Attachment Cleaner Power Tool regularly to keep the database sizes manageable.

The Test Attachment Cleaner however is a command-line tool which requires the Collection Uri and the Team Project Name among several other parameters so I first needed to write a script (I chose PowerShell) to query TFS for all the Collections and Projects and call the Cleaner for each. I then needed to configure this script (and therefore the Cleaner) to run as a scheduled task and I needed to specify which user the task would run as.

The easiest answer would be to run the task as a user who has TFS Server Administrator privileges to ensure the Cleaner has access to find and delete attachments in every project in every collection but that would be overkill. I couldn’t find any existing documentation on the minimum privileges required by the Cleaner so I started with a user with zero TFS privileges and repeatedly executed the scheduled task, granting each permission mentioned in each successive error message until the task completed successfully.

For deleting attachments by extension and age I found the minimum permissions required were the following three, all at the Team Project level:

  • View test runs
  • Create test runs (non-intuitively, this permission allows attachments to be deleted)
  • View project-level information

For deleting attachments based on linked bugs (something I didn’t try) I suspect the “View work items in this node” permission would also be required at the root Area level.

Having determined these permissions, I needed to apply them across all the Team Projects but it appears the only out-of-the-box way to set this permissions is via the Team Explorer user interface which becomes rather tedious after the first few projects. Instead I scripted the permission granting too via the TFS API and I’ve published some PowerShell cmdlets to make this easier if anyone else needs to do the same.

Rules to Customising a .NET Build

It doesn’t take long before any reasonable software project requires a build process that does more than the IDE’s default Build command. When developing software based on the .NET platform there are several different ways to extend the build process beyond the standard Visual Studio compilation steps.

Here is a list for choosing the best place to insert custom build steps into the process, with the simplest first and the least desirable last:

  1. Project pre- and post-build events: There is a GUI, you have access to a handful of project variables, can perform the same tasks as a Windows Batch script, and still works with Visual Studio’s F5 build and run experience. Unfortunately only a failure of the last command in a batch will fail the build and the pre-build event happens before project references are resolved so avoid using it to copy dependencies. Continue reading

Override the TFS Team Build OutDir property

Update: with .NET 4.5 there is an easier way.

A very common complaint from users of Team Foundation Server’s build system is that it changes the folder structure of the project outputs. By default Visual Studio puts all the files in each project’s respective /bin/ or /bin/<configuration>/ folder but Team Build just uses a flat folder structure putting all the files in the drop folder root or, again, a /<configuration>/ subfolder in the drop folder, with all project outputs mixed together.

Additionally because Team Build achieves this by setting the OutDir property via the MSBuild.exe command-line combined with MSBuild’s property precedence this value cannot easily be changed from within MSBuild itself and the popular solution is to edit the Build Process Template *.xaml file to use a different property name. But I prefer not to touch the Workflow unless absolutely necessary.

Instead, I use both the Solution Before Target and the Inline Task features of MSBuild v4 to override the default implementation of the MSBuild Task used to build the individual projects in the solution. In my alternative implementation, I prevent the OutDir property from being passed through and I pass through a property called PreferredOutDir instead which individual projects can use if desired.

The first part, substituting the OutDir property for the PreferredOutDir property at the solution level is achieved simply by adding a new file to the directory your solution file resides in. This new file should be named following the pattern “before.<your solution name>.sln.targets”, eg for a solution file called “Foo.sln” then new file would be “before.Foo.sln.targets”. The contents of this new file should look like this. Make sure this new file gets checked-in to source control.

The second part, letting each project control its output folder structure, is simply a matter of adding a line to the project’s *.csproj or *.vbproj file (depending on the language). Locate the first <PropertyGroup> element inside the project file that doesn’t have a Condition attribute specified, and the locate the corresponding </PropertyGroup> closing tag for this element. Immediately above the closing tag add a line something like this:

<OutDir Condition=" '$(PreferredOutDir)' != '' ">$(PreferredOutDir)$(MSBuildProjectName)\</OutDir>

In this example the project will output to the Team Build drop folder under a subfolder named the same as the project file (without the .csproj extension). You might choose a different pattern. Also, Web projects usually create their own output folder under a _PublishedWebSites subfolder of the Team Build drop folder, to maintain this behaviour just set the OutDir property to equal the PreferredOutDir property exactly.

You can verify if your changes have worked on your local machine before checking in simply by running MSBuild from the command-line and specifying the OutDir property just like Team Build does, eg:

msbuild Foo.sln /p:OutDir=c:\TestDropFolder\

Queue another Team Build when one Team Build succeeds

Update: with Team Build 2013 you can even pass parameters to queued builds.

I have seen several Team Foundation Server environments where multiple build definitions exist in a single project and need to executed in a particular order. Two common techniques to achieve this are:

  • Queue all the builds immediately and rely upon using a single build agent to serialize the builds. This approach prevents parallelization for improved build times and continues to build subsequent builds even when one fails.
  • Check build artifact(s) into source control at the end of the build and let this trigger subsequent builds configured for Continuous Integration. This approach can complicate build definition workspaces and committing build artifacts to the same repository as code is not generally recommended.

As an alternative I have developed a simple customization to TFS 2010’s default build process template (the DefaultTemplate.xaml file in the BuildProcessTemplates source control folder) that allows a build definition to specify the names of subsequent builds to queue upon success. It only requires two minor changes to the Xaml file. The first is a line inserted immediately before the closing </x:Members> element near the top of the file:

    <x:Property Name="BuildChain" Type="InArgument(s:String[])" />

The second is a block inserted immediately before the final closing </Sequence> at the end of the file

    <If Condition="[BuildChain IsNot Nothing AndAlso BuildChain.Length &gt; 0]" DisplayName="If BuildChain defined">
      <If.Then>
        <If Condition="[BuildDetail.CompilationStatus = Microsoft.TeamFoundation.Build.Client.BuildPhaseStatus.Succeeded]" DisplayName="If this build succeeded">
          <If.Then>
            <Sequence>
              <Sequence.Variables>
                <Variable x:TypeArguments="mtbc:IBuildServer" Default="[BuildDetail.BuildDefinition.BuildServer]" Name="BuildServer" />
              </Sequence.Variables>
              <ForEach x:TypeArguments="x:String" DisplayName="For Each build in BuildChain" Values="[BuildChain]">
                <ActivityAction x:TypeArguments="x:String">
                  <ActivityAction.Argument>
                    <DelegateInArgument x:TypeArguments="x:String" Name="buildChainItem" />
                  </ActivityAction.Argument>
                  <Sequence DisplayName="Queue chained build">
                    <Sequence.Variables>
                      <Variable Name="ChainedBuildDefinition" x:TypeArguments="mtbc:IBuildDefinition" Default="[BuildServer.GetBuildDefinition(BuildDetail.TeamProject, buildChainItem)]"/>
                      <Variable  Name="QueuedChainedBuild" x:TypeArguments="mtbc:IQueuedBuild" Default="[BuildServer.QueueBuild(ChainedBuildDefinition)]"/>
                    </Sequence.Variables>
                    <mtbwa:WriteBuildMessage Message="[String.Format(&quot;Queued chained build '{0}'&quot;, buildChainItem)]" Importance="[Microsoft.TeamFoundation.Build.Client.BuildMessageImportance.High]" />
                  </Sequence>
                </ActivityAction>
              </ForEach>
            </Sequence>
          </If.Then>
        </If>
      </If.Then>
    </If>

You can see the resulting DefaultTemplate.xaml file here. After applying these changes and checking-in the build process template file you can specify which builds to queue upon success via the Edit Build Definition windows in Visual Studio:

The new BuildChain build process parameter

You can specify the names of multiple Build Definitions from the same Team Project each on a separate line. When the first build completes successfully, all builds listed in the BuildChain property will then be queued in parallel and processed by the available Build Agents. No checking is currently done for circular dependencies so be careful not to chain a build to itself directly or indirectly and create an endless loop.

Fail a build when the warning count increases

I like to set the “Treat warnings as errors” option in all my Visual Studio projects to “All” to ensure that the code stays as clean and maintainable as possible and issues that may not be noticed until runtime are instead discovered at compile time. However, on existing projects with a large number of compiler warnings already present in the code base, turning this option on will immediately fail all the CI builds and either create a lot of work for the team before the builds are passing again or the team will start ignoring the failed builds. Neither is a desirable situation.

On the other hand, unless someone is actively monitoring the warnings and notifying the team when new compile warnings are introduced, the technical debt is just going to increase. To address this I thought it would be useful to extend the default build process template in TFS 2010 to compare the current build’s warning count with the warning count from the previous build and fail the build if the number of warnings has increased. The Xaml required for this can be found here. Hopefully this strategy will result in the team slowly decreasing the warning count down to zero and then the “Treat warnings as errors” option can be enabled to prevent new compiler warnings being introduced to the code base.

At the moment this is a very naive implementation – if an increase in warnings fails one build, subsequent builds will pass unless the warning count increases again. I have two ideas for improving this:

  1. Compare the current warning count against the minimum warning count of all previous builds.
  2. Fail if the minimum warning count has not decreased within a specified time period (eg two weeks).

If I implement either of these two ideas, I’ll update this post but they should both be quite easy to do-it-yourself.