Since Team Foundation Server 2010, Microsoft has shipped the Test Manager product which enables testers to document manual test cases and later automate them (or just automated them from the beginning). When used to its full potential with other TFS components like Team Build and Lab Management, managing Test Cases with TFS provides a great way to improve and verify the quality of the software you deliver.
In Test Manager, or more accurately Visual Studio, automation for a Test Case is associated via two primary identifiers:
- The file name of the .NET Assembly containing the relevant Test Method, eg “CodedUITestProject1.dll”
- The namespace- and class-qualified name of the Test Method, eg “CodedUITestProject1.CodedUITest1.CodedUITestMethod1”
After associating one or more Test Methods with Test Cases you may later need to refactor your code resulting in many Test Methods moving to a new class or namespace, or even a new assembly name. Unfortunately, after this refactoring effort, Microsoft Test Manager will no longer be able to locate the Test Methods associated with affected Test Cases and the tests will fail to run.
There are some options for fixing this, of varying effectiveness:
- Undo the refactoring – pointless but technically an option.
- Manually open each one of the Test Cases in Visual Studio and re-associate the automation – tedious for larger numbers of tests.
- If only the assembly name has changed, open the list of Test Cases in Excel, include the “Automated Test Storage” column, and bulk update the values.
BUT, if the namespace or class has changed, the fix could be slightly more complicated…
You may be able to use Excel with the “Automated Test Name” column included and you may find it works but there is also another column to consider, usually hidden, called “Automated Test Id”. I know this column is used by the TCM command-line tool to automatically import Test Methods into Test Cases and it may be used in other areas of Visual Studio and TFS.
In tcm.exe, the Id is based on the Test Name and used to detect if a Test Method has already been imported and to prevent the creation of duplicate Test Cases. Gautam Goenka’s blog post on associating automation programmatically demonstrates the simple Id generation algorithm used.
As per my usual style, I’ve published a PowerShell script, called “Rename-TestCaseAutomation” on GitHub Gist, which will locate all the Test Cases with associated automation using a specified Assembly Name and/or Class Name prefix and update them with a provided new Assembly Name or Class Name Prefix, and update the Id appropriately too. Here are two simple example scenarios for using it:
Rename the Assembly for all Test Cases across two different Team Projects:
Rename-TestCaseAutomation http://localhost:8080/tfs/DefaultCollection MyProject,YourProject -TestStorage CodedUITestProject1.dll -NewTestStorage FooTests.dll
Move all the Tests Methods in the “FooTests.Customer” Class to to the “FooTests.Client” Class:
Rename-TestCaseAutomation http://localhost:8080/tfs/DefaultCollection MyProject -TestNamePrefix FooTests.Customer -NewTestNamePrefix FooTests.Client
Hopefully someone else finds this useful.
I recently completed a client engagement that involved building a suite of automated user interface tests for a critical internal web application. The web application has existed for several years and is built to work with Internet Explorer (the client’s SOE browser) and the client has also invested heavily in the Microsoft ALM stack including Team Foundation Server, Test Manager, and some advanced capabilities of Lab Management. Given this background, Coded UI was a suitable technology choice for driving the user interface from tests.
Initially my team and I wanted to take a code-first approach to Coded UI, something similar to the Page Object pattern commonly used in conjunction with Selenium-based testing. However, given that the test team would be responsible for maintaining the test suite, not the developers, the client pushed for us to build a solution using the Coded UI Test Builder tool to record the tests interactively. From experience I’ve found that record-and-replay testing approaches tend to be fragile and difficult to maintain so we found a compromise, a hybrid solution.
The design involved having a single UIMap (“.uitest” file) corresponding to each of the individual pages (or popup windows) in the application. Within each UIMap, we recorded discrete, almost atomic actions, as individual methods (eg “enter text in search box” and “click search button” would be separate methods, recorded separately) and similarly with asserts. We would then hand-code the actual test methods as as series of calls to each of the actions and asserts on the relevant UI Maps, interspersed with control flow code as needed. As an intentional by-product of naming our UIMaps and methods clearly, the hand-coded test methods read very similarly to original manual steps described in each Test Case. Coincidentally, a lot of this approach aligns with the recommended practices.
With this hybrid approach, whenever a part of the web application changes, instead of needing to re-record large chunks of actions and tests, only the individual affected pieces need to be updated. The problem with the Coded UI recoding model in general though is dealing with HTML Tables and the application we were testing used Tables to display grids of data and for selecting data on almost every page. When recording actions against a Table with the Coded UI Test Builder it results in code that will later match a cell based on its row and column index or its cell content, or both. This is fine for single-cell actions and assertions but when you need to locate, for example, the cell in the first column containing a specific account number and then click the corresponding cell in the third column on the same row, Coded UI recordings don’t help. In fact, the hand-coded equivalent of this scenario using the Coded UI APIs directly isn’t great either.
As we progressed through the test cases, we realised the Table scenario needed a good solution. We tried three different approaches over the course of the project and improved incrementally on each based on what we learned. In the end we had a strongly-typed table abstraction that kept the test methods clean and the UIMaps maintainable. The Coded UI Test Builder was still used to record to selection of the HTML <table> element within the page, and we kept the column mapping simple in the UIMap partial class, so we didn’t prevent the testers from easily updating the tests with the application.
I have published the table abstraction code we used on GitHub, along with a basic DemoWeb application and an accompanying Coded UI test to demonstrate how we use it. Here is the Page Object pattern in use where a particular row structure is defined along with some domain-specific Find and Assert methods.