microsoft/vsts-work-item-migrator

WiMigrator is a command line tool for migrating work items between VSTS/TFS projects

Introduction

WiMigrator is a command line tool designed with the following goals in mind:

• Migrate work items from one Azure DevOps/TFS project to another
• Real world example of how to use the WIT REST APIs
• Cross platform support

Features

• Migrate the latest revision of a work item or set of work items based on the provided query, including:
  • Work item links (for work items within the query results set)
  • Attachments
  • Git commit links (link to the source git commit)
  • Work item history (last 200 revisions as an attachment)
  • Tagging of the source items that have been migrated

Getting Started

Requirements

• Source Project on Azure DevOps or TFS 2017 Update 2 or later
• Target Project on Azure DevOps or TFS 2018 or later
• Personal access tokens or NTLM for authentication
• Bypass rules or Project Collection Administrator permissions required on target project
• Process metadata should be consistent between the processes
  • Limited field mapping support is provided to map fields from the source to target account
  • Area/Iteration paths can be defaulted to a specific value when they don't exist on the target

Running

WiMigrator supports the following command line options:

• --validate validates that the metadata between the source and target projects is consistent
• --migrate re-runs validation and then migrates the work items

Migration runs in two parts:

• Validation
  • Configuration settings
  • Process metadata is consistent between projects
  • Identifies any work items that were previously migrated
• Migration
  • Phase 1: Work item fields
  • Phase 2: Attachments, links, git commit links, history, target move tag
  • Phase 3: Source move tag

A sample configuration file is provided with documentation of all the settings.

Execution example:

dotnet run --validate configuration.json

HOWTO Video

Limitations:

• Artifact links (other than git) are not migrated
• Board fields are not migrated
• Test artifacts (e.g. test results) are not migrated

Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.microsoft.com.

When you submit a pull request, a CLA-bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.

vsts-work-item-migrator's Issues

Work Item Creation fails with 400 response code: "VS403426: IdentityRef type is not accepted for this request."

When I tried this code a month ago for a VSTS to VSTS WIT migration everything worked great. When I went to try it again this week using the same configuration I get a failure with a 400 response back for all attempts to create any type of WIT. Did something change with the VSTS API that has broken this project? Here is an example request url and response (the body contains the WIT field data as I'd expect and have omitted for privacy).

METHOD: PATCH
URI: /MigrationTest1/_apis/wit/workItems/$Epic?bypassRules=True&suppressNotifications=True&api-version=4.0

RESPONSE CODE:
400

RESPONSE BODY:
{"count":1,"value":{"Message":"VS403426: IdentityRef type is not accepted for this request."}}

This happens for every WIT that is in my source query (all different types, Epics, Features, Stories, Tasks, etc). I also tried creating a new VSTS project for a destination just to see if I needed a clean slate, but it made no difference. Anyone know what's going on here? Does this program still work for anyone else?

Querying for already existing work items seems to ignore project

We are in the situation where we have once successfully migrated a set of work items to a new team project in the same collection as the original project. When we now want to do this action again to another team project at a later time (short story: think of it as a way of "baselining" a set of work items and their states), the tool claims that a number of work items already have been migrated. So:

Migration 1: Original ==> Project 1, 10000 work items to migrate (success)
Migration 2: Original ==> Project 2, 11000 work items to migrate (only the 1000 new ones are migrated)

It seems as if the status check whether a work item already has been migrated is cross-project, and then returns those work items that have been migrated to the previous project.

Add the web link to the work item

We add the json link, we should add an option to add the web link too.

iterations duration

I am migrating user stories from vsts -> tfs but i am not able to migrate time duration of iterations, is it possible to perform this or this migrator cant do that?

Invalid Area/Iteration Path

Hi everyone. When I ran the Validation, I got a warning saying my work items have been skipped due to an invalid area/iteration path or type and will not be migrated. Any solution ideas? Thanks in advance

Dave

Troubleshooting Missing Attachments on Migration

Running this on a Mac everything seems to be be migrating well except for the attachments.

In WorkItemTrackingHelpers.cs

The following code is getting hit. I keep getting stuck around line 136 in which I'm getting a 400 bad request, unfortunately I can't figure out how to get a good return from this service.

        public async static Task<AttachmentReference> CreateAttachmentChunkedAsync(WorkItemTrackingHttpClient client, VssConnection connection, MemoryStream uploadStream, int chunkSizeInBytes)
        {
            // it's possible for the attachment to be empty, if so we can't used the chunked upload and need
            // to fallback to the normal upload path.
            if (uploadStream.Length == 0)
            {
                return await CreateAttachmentAsync(client, uploadStream);
            }

            var requestSettings = new VssHttpRequestSettings
            {
                SendTimeout = TimeSpan.FromMinutes(5)
            };
            var httpClient = new HttpClient(new VssHttpMessageHandler(connection.Credentials, requestSettings));

            // first create the attachment reference.  
            // can't use the WorkItemTrackingHttpClient since it expects either a file or a stream.
            var attachmentReference = await RetryHelper.RetryAsync(async () =>
            {
                var request = new HttpRequestMessage(HttpMethod.Post, $"{connection.Uri}/_apis/wit/attachments?uploadType=chunked&api-version=3.2");
                var response = await httpClient.SendAsync(request);

                if (response.IsSuccessStatusCode)
                {
                    return await response.Content.ReadAsAsync<AttachmentReference>();
                }
                else
                {
                    var exceptionResponse = await response.Content.ReadAsAsync<ExceptionResponse>();
                    throw new Exception(exceptionResponse.Message);
                }
            }, 5);

Further more I can't find the documentation for this api-version (current set to 3.2) to support this problem. Is it even necessary to create reference for these attachments any more?

Add area path and iteration path mapping

If the area path not existing to target project, can we not just migrate all of them to "default-area-path", provide some mapping options?

For example:
In configuration.json, the "default-area-path": A
In source project, the area path list is {B, C, D}
In target project, the area path list is {A, B, E}

Is it possibility to migrate the area path with the relation B -> B, C -> E, D -> A?

Error uploading attachments of different file types

I'm running the migrator from one project on VSTS to another project on another organizations VSTS.
The validation phase runs fine, the migration phase has a ton of critical errors.
Almost all of them looks something like the following:

[Error   @14.37.26.992] Unable to upload attachment front end validation.mov for source work item 90389 to the target account
System.Net.Http.UnsupportedMediaTypeException: No MediaTypeFormatter is available to read an object of type 'ExceptionResponse' from content with media type 'text/html'.
   at Common.RetryHelper.RetryAsync[T](Func`1 function, Func`3 exceptionHandler, Int32 retryCount, Int32 secsDelay) in C:\Users\jkc\Downloads\vsts-work-item-migrator\Common\RetryHelper.cs:line 78
   at Common.RetryHelper.RetryAsync[T](Func`1 function, Int32 retryCount, Int32 secsDelay) in C:\Users\jkc\Downloads\vsts-work-item-migrator\Common\RetryHelper.cs:line 18
   at Common.WorkItemTrackingHelpers.CreateAttachmentChunkedAsync(WorkItemTrackingHttpClient client, VssConnection connection, MemoryStream uploadStream, Int32 chunkSizeInBytes) in C:\Users\jkc\Downloads\vsts-work-item-migrator\Common\WorkItemTrackingHelpers.cs:line 131
   at Common.Migration.AttachmentsProcessor.UploadAttachmentFromSourceRelation(IMigrationContext migrationContext, IBatchMigrationContext batchContext, WorkItem sourceWorkItem, WorkItemRelation sourceRelation, Int32 maxAttachmentSize) in C:\Users\jkc\Downloads\vsts-work-item-migrator\Common\Migration\Phase2\Processors\AttachmentsProcessor.cs:line 184

I've checked and it happens regardless of the size of the attachment. It seems text files work fine, but all files that are pictures/movies/excel files give this error.

Is there anything I can do?

TF400813: Resource not available for anonymous access. Client authentication required.

I am unable to get the migrator running using the following settings:

"source-connection": {
    "account": "https://myAccount.visualstudio.com/",
    "project": "OldProject",
    "access-token": "PersonalAccessTokenWithFullAccess",
    "use-integrated-auth": "false"
  },
  "target-connection": {
    "account": "https://myAccount.visualstudio.com/",
    "project": "NewProject",
    "access-token": "PersonalAccessTokenWithFullAccess",
    "use-integrated-auth": "false"
  },

I used the same PAT for the source and target connection because they are under the same account, just in a different project.

The access-token property is not available in the logging, but that may be by design.

---

Output:

new work items found:                      0
existing work items found:                 0
existing work items validated for phase 1: 0
existing work items validated for phase 2: 0
Waiting for query to retrieve work items to be validated...
[Info   @16.36.10.326] Starting validation
[Info   @16.36.10.328] Starting configuration validation
[Info   @16.36.10.390] Starting configuration validation for: Source account permissions
[Info   @16.36.10.391] source-post-move-tag is specified, checking write permissions on the source project
[Error   @16.36.10.592] Unexpected error:
Microsoft.VisualStudio.Services.Common.VssUnauthorizedException: TF400813: Resource not available for anonymous access. Client authentication required.
   at Microsoft.VisualStudio.Services.Common.VssHttpMessageHandler.<SendAsync>d__17.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.Common.VssHttpRetryMessageHandler.<SendAsync>d__4.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1.ConfiguredTaskAwaiter.GetResult()
   at System.Net.Http.HttpClient.<FinishSendAsyncBuffered>d__58.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<SendAsync>d__49.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.<SendAsync>d__46`1.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.Location.Client.LocationHttpClient.<GetConnectionDataAsync>d__6.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.WebApi.Location.VssServerDataProvider.<ConnectAsync>d__41.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.WebApi.Location.VssServerDataProvider.<EnsureConnectedAsync>d__39.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.WebApi.Location.VssServerDataProvider.<GetAuthorizedIdentityAsync>d__5.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.VisualStudio.Services.WebApi.VssConnection.get_AuthorizedIdentity()
   at Common.Validation.ValidationHelpers.<CheckConnection>d__5.MoveNext() in C:\dev\temp\vsts-work-item-migrator\Common\Validation\ValidationHelpers.cs:line 26
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Common.Validation.ValidateSourcePermissions.<Validate>d__5.MoveNext() in C:\dev\temp\vsts-work-item-migrator\Common\Validation\Configuration\ValidateSourceClient.cs:line 29
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Common.Validation.Validator.<ValidateConfiguration>d__6.MoveNext() in C:\dev\temp\vsts-work-item-migrator\Common\Validation\Validator.cs:line 104
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Common.Validation.Validator.<Validate>d__5.MoveNext() in C:\dev\temp\vsts-work-item-migrator\Common\Validation\Validator.cs:line 26
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at WiMigrator.CommandLine.<ExecuteValidation>d__8.MoveNext() in C:\dev\temp\vsts-work-item-migrator\WiMigrator\CommandLine.cs:line 84
[Info   @16.36.10.614] Summary of errors and warnings:
[Error   @16.36.10.625] Unexpected error:. TF400813: Resource not available for anonymous access. Client authentication required.. TF400813: Resource not available for anonymous access. Client authentication required.```

Validation fails with "Unexpected Error"

Work item ID used in destination project breaks Parent/Child work items link.

Hi, I've successfully moved all work items and after creating and new team (same as source) and assigned rights all looked good. However, most iterations have an title group called "unparented" and in it are all the unlinked work items. After investigation it seems that the work ID was already used in the destination so a new work ID was created. This breaks the linking, is there a section I missed? If not any programatic way to relink?

Changing the Process Model

Will this tool assist in migrating data between collections with disparate process models?

In other words, I'd like to switch from XML to Inheritance.

Migrating Identities from TFS to VSTS

More of a design question here.

Looking into adding a feature which can replace identities from local AD accounts in TFS to ones in Azure AD accounts in VSTS.

High-level design was to introduce another key into the configuration files.

migrating-identities = [
{"source": "anger1", "target": "[email protected]"},
{"source": "bill", "target": "[email protected]"}
]

The processors are the part I'm making sense of still though.

1. Make a new processor similar to the IdentityPreProcessor, maybe MigrateIdentityPreProcessor.
2. Override IsEnabled to look for any list in the migrated-identities field. Any items in this lists would be the litmus test for enabled/true.
3. Read through source WIs and when I come across one of the migrated-identities replace it with the new one.
4. Have it run prior to IdentityPreProcessor so it still can have the affect to create those identities.

Does this seem like a sensible approach?

Support work item type mapping

Currently, you can't migrate work items from a team project created with one process template to another created with a different process template. Support for field mapping exists, but not work item type mapping. Suggestion to add support in vsts-work-item-migrator to allow mappings between "Requirement", "User Story", "Product Backlog Item" work item types (the RequirementsCategory types).

Discussion not migrating for VSTS to VSTS Migration

We are performing a VSTS-->VSTS Migration.
All source data appears to be migrating well except discussion which is always blank in the destination system.

unable to read the migration query

Hi,

Getting the above error when running the tool. I left the default value in the query section. I have Project Collections Admin on both sides.
From config
"query": "Shared Queries/Website/All Tasks"
I also changed the values in the query, and pulled latest. Am I missing something? Here is the error from the log

[Error @14.12.34.428] Validation error:
Common.Validation.ValidationException: Unable to read the migration query ---> Common.RetryPermanentException: Permanent error for ba8bb07e-fce6-4485-896d-7ebbe5550bbf, not retrying ---> Microsoft.VisualStudio.Services.Common.VssServiceException: TF401243: The query Shared Queries/Website/All Tasks does not exist, or you do not have permission to read it.

Comments are not migrated

Hi,
I am trying to migrate work items from TFS to VSTS. Post migration, I could notice that none of the comments got migrated. Is it known issue with VSTS work item migrator?

Thanks
Girish

Git commit links point to the wrong account

When migrating git commit links, when converting the git commit artifact URI to a hyperlink it is using the target account instead of the source account.

Unable to migrate in Azure DevOps Server 2019: VS403417

We are testing migration in Azure DevOps Server 2019 and are having trouble making it work. For every workitem, we receive the error
VS403417: ChangedBy value cannot be empty when BypassRules is specified

When debugging, it seems to us that Microsoft has changed returning "identity fields" from a string {first name} {last name} <{user-id}> to a complete Microsoft.VisualStudio.Services.WebApi.IdentityRef. When we "hack" in Common\Migration\Phase1\WitBatchRequestGenerators\BaseWitBatchRequestGenerator.cs to instead return the old type of value using the code below, it works again:

if(sourceField.Value.GetType() == typeof(Microsoft.VisualStudio.Services.WebApi.IdentityRef)){
  var identity = (Microsoft.VisualStudio.Services.WebApi.IdentityRef)sourceField.Value;
  string updatedIdentityFieldValue = identity.DisplayName + " <" + identity.UniqueName + ">";
  KeyValuePair<string, object> updatedField = new KeyValuePair<string, object>
}

We know this not a viable solution to the problem. But we wanted to notify you that there might be a general problem using this tool in Azure DevOps Server 2019, and we feel we are not proficient enough in you tool to make the correct changes ourselves.

(Btw, we are still using TFS 2018 in our production environment, so the migration works fine there without using this "hack").

Could not find source project from area path

Hello,
I'm trying to run a migration of work items from one project to another, on two different devops accounts.
When I run the validation I get the following error:
[Info @08.35.44.264] Checking metadata for work item type Test Case [Info @08.35.44.264] Work item type Test Case validation completed successfully [Error @08.35.44.534] Validation error:. Could not find source project from area path Source Project\Project Area Path for work item with id 72580 [Info @08.35.44.539] Summary of errors and warnings: [Error @08.35.44.539] Validation error:. Could not find source project from area path Source Project\Project Area Path for work item with id 72580. Could not find source project from area path Source Project\Project Area Path for work item with id 72580

Looking in the code the following comment is at the validation error:
// This is a fatal error because this implies the query is cross project which we do not support, so bail out immediately

I'm not sure what this means, as the examples I've seen are all from one project to another.

I've tried making the same area path structure on the target project as it suggested by Mohamed Radwan in a comment on his guide. But it didn't change anything.

Could not find comment in relation hyperlink to source work item on target work item with id

When running the WiMigrator I am finding that validation passes, migration phase 1 passes, however phase 2 ends with the following unexpected error.

Could not find comment in relation hyperlink to source work item on target work item with id: 5366. Expected source work item id: 216112

I've tried a few different configuration changes in the configuration.json like re-mapping some problematic fields but still seem to be getting the error.

An item with the same key has already been added

I have some work items already in the target project and now I am trying to migrate the old work items. In this Validation went successfully but after executing the migrate command I am getting the following error message “An item with the same key has already been added.” I try to change the work item titles to be unique.I also try creating a new project and run the migration to the newly created project. But still same issue. Could you please help on this, what I am doing wrong.

Workitem links not properly migrated

Expected behavior:

Links between workitems in the source project are restored in the target project

(Image is from source project)

Actual behavior:

Links between workitems in the source project are placed as hyperlinks in the target project

(Image is from target project)

The work item on the target project does contain a hyperlink to the ID of the work item in the source project:

Summary:

I just migrated our workitems from one VSTS project to another one (within one account), but the links were not properly migrated. If I look at the backlog on the source project, I see that most stories and bugs have children:

If I look at the backlog on the target project, I see the workitems, but the children are not there:

Existing work item retrieval ignores filtering on target project

Note: This item is similar to issue #47 "Querying for already existing work items seems to ignore project".

To repro:

1. We are migrating from on-premise TFS to Azure DevOps.
2. Set the post-move-tags the config file to empty and set the skip-existing param to false
3. Create a test (i.e. Test) Azure DevOps project
4. Create a query to use in the on-premise TFS project that migrates 1/3 the work items
5. Run the migration tool to migrate the items into the Test Azure project (notice the # of existing items display as 0 in the target project during the migration)
6. Create a final (i.e. Final) Azure DevOps project
7. Update the query in TFS to migrate all the items
8. Run the migration tool again on the Final Azure project (notice how the # of existing items display as the 1/3 amount in the target project during the migration)

Expected behavior:
The migration tool should read any existing migrated work items from the target project only (as opposed to any work items in the target Azure DevOps organization)

Actual behavior:
The migration tool appears to be reading work items from any project (e.g. the Test) project rather than the target project (e.g. Final)

Migration fails when project name has spaces

Hello, I've been trying to migrate a project that has spaces in its name. The error the migration tool was giving me was something like "Could not found area path for project ...".

The project had spaces in its name, and since the default area path is the project name, the only way of changeing it was by renaming the project.

So I did and it worked.

Regards

Re-linking test work items

The migrator will bring over test plans, suites, and cases, but it won't relate them to each other correctly because they are not linked like other work items. For example, a test case created independently won't appear in a test suite even if that suite is linked as the parent.

This is known behavior but the migrator is of limited utility without some means to migrate the test work item relationships. I took a quick look at the REST API but didn't see anything which could be used to solve this. Any ideas? @sferg-msft

Work item number is not sync with source?

Field Replacements not working

I want to map five source fields to target fields that I specified in a template called ScrumCustomized. The migration is successful but these fields have not being populated. Any ideas? Am I missing something?

"field-replacements": {
"Scrum.v3.EstimatedEffort": { "field-reference-name": "ScrumCustomized.EstimatedEffort" },
"Custom.Confidence": { "field-reference-name": "ScrumCustomized.Confidence" },
"tfl.BusinessSummaryInfo": { "field-reference-name": "ScrumCustomized.BusinessSummaryInfo" },
"tfl.BusinessRationale": { "field-reference-name": "ScrumCustomized.BusinessRationale" },
"Tfl.ProjectCode": { "field-reference-name": "ScrumCustomized.ProjectCode" }
},

Unable to read the migration query

Hi All,

I am migrating Work Items from TFS 2017 to Azure DevOps.

It is working properly with some projects but I am getting error on a project:

"Unable to read the migration query"

in the log:

[Error @07.22.29.275] Validation error: Common.Validation.ValidationException: Unable to read the migration query ---> Common.RetryPermanentException: Permanent error for 683c5b2c-649c-4dd1-9ca1-49c22ee052c0, not retrying ---> Microsoft.VisualStudio.Services.WebApi.VssServiceResponseException: Page not found.

I created the affected Query under the Shared Queries and all another settings are similar with the another projects.

The project name is contain slash "/ ", maybe the Migrator could not handle this character?

Regards,
Gabor

Migration fails with "Object reference not set to an instance of an object."

When default-iteration-path or default-area-path do not exist on the target project migration fails with uninformative error message - "Object reference not set to an instance of an object."

Instead - migrator should validate default-iteration-path or default-area-path and display meaningful errors if they don't.

Does this work cross organization in the latest Azure DevOps ?

We're trying to migrate some items from a project within one org. to another project in another org.

Is it possible to Migrate work items from TFS 2010 to TFS 2018 or Latest?

Hello,

Can you please let me know if it possible to migrate work-items & it's flow to TFS 2018/2019?

Is there anyway to get this done?
Thank you,
RR

failed for this reason: BadRequest

Hi Team,

I am getting the issue BadRequest , when trying to migrate from VSTS to another VSTS. Validation is getting passed.

A video on how to use the tool

I created a video on how to use the tool so anyone can get quickly overview of it, I thought to make a pull request to add it to the readme then I thought it's better to leave that to you if you think it could help.
Thanks!
https://www.youtube.com/watch?v=aHbiLYUfomc&feature=youtu.be

work item

Bad Request - IdentityRef type is not accepted for this request.

Hi,

I am running the tool to migrate from one org to another org. Process templates are both Scrum. When I run the tools I get the following error for each work item I try to migrate.

{"Message":"VS403426: IdentityRef type is not accepted for this request."}

I am not finding a solution that will work for me. Can anyone shed some light on the issue I am having?

Thanks,

How update only area and iteration path

Hello,
I migrated my work items but i forgot create in the target the correct iterations

Now, I created the correct iterations and i want only update the area and iteration path

How is possible do that?

This is my config

{
"source-connection": {
"account": "https://source.visualstudio.com",
"project": "projectSource",
"access-token": "...................",
"use-integrated-auth": "false"
},
"target-connection": {
"account": "https://target.visualstudio.com",
"project": "projectTarget",
"access-token": "........................",
"use-integrated-auth": "false"
},
"query": "My Queries/AllWorkItems",
"heartbeat-frequency-in-seconds": 30,
"query-page-size": 20000,
"parallelism": 1,
"max-attachment-size": 62914560,
"link-parallelism": 1,
"attachment-upload-chunk-size": 1048576,
"skip-existing": false,
"move-history": true,
"move-history-limit": 200,
"move-git-links": true,
"move-attachments": true,
"move-links": true,
"skip-work-items-with-type-missing-fields": false,
"skip-work-items-with-missing-area-path": false,
"skip-work-items-with-missing-iteration-path": false,
"clear-identity-display-names": false,
"ensure-identities": false,
"include-web-link": true
}

Not working with non secure TFS server

Hi Team,

As I have tried migrating work Items with WIMigrator from TFS to VSTS, but getting error regarding permission, but I am having administrator access on TFS server, my TFS sever is having Non secure site,

Please suggest if we can use the tool for non secure TFS server.

log file & configuration file attached below.

WiMigrator_Migrate_2018-09-27_08-47-28.log

Migration fails at the end of phase 2, when applying tags.

I'm migrating from TFS 2017 update 3.
Migration fails at the end of phase 2, when applying tags. It returns: Status Code: 404.
WiMigrator_Migrate_2018-06-07_14-45-19.log
WitBatchRequestsFromBatch4WithFailure2018-06-07_14-50-33-14.log
WitBatchRequestsFromBatch3WithFailure2018-06-07_14-50-32-09.log
WitBatchRequestsFromBatch2WithFailure2018-06-07_14-50-31-056.log
WitBatchRequestsFromBatch1WithFailure2018-06-07_14-50-29-806.log

Account Uri requires trailing '/'

I encountered a problem when copying attachments in on-premises TFS. For every attachment that was to be copied, I received the following warning:
[Warning @15.22.59.449] Exception caught for b55743bd-e3a1-4663-84ee-d9c54534a5de: System.Net.Http.UnsupportedMediaTypeException: No MediaTypeFormatter is available to read an object of type 'ExceptionResponse' from content with media type 'text/html'. at System.Net.Http.HttpContentExtensions.ReadAsAsync[T](HttpContent content, Type type, IEnumerable1 formatters, IFormatterLogger formatterLogger, CancellationToken cancellationToken)
at Common.WorkItemTrackingHelpers.<>c__DisplayClass11_0.<b__0>d.MoveNext() in C:\repos\WiMigrator\Common\WorkItemTrackingHelpers.cs:line 138`

The problem was that the combined REST API address lacked a '/' between account Uri and '_apis/...', thus making the call to upload attachments failed. When this was noticed during debugging, it was a simple task to add the trailing '/' to make it work, but I believe this could be solved by either documenting it in sample-configuration.json, or -better - by automatically adding a trailing '/' if missing.

Only the last history/discussion item moved over on a VSTS to TFS Migration

Expected:

As you can see, there is a large discussion history on the original work item. When I run the tool, I expected it to copy all of these comments.

Actual:

As you can see, the only item copied was the last one in the chain. I'm not sure if this is intentional or not, but I do think it's important if it copies over the history to copy over all of it.

Migration phase 1 is failing on updating work item (links)

After migration phase 1 creates the new work items, it attempts to link them and fails with a rather cryptic TF401349 error. Brief search on the TFS error code turned up very little of value. Here are the error lines from the log output:

[Error   @14.03.01.691] Exception during batch 1: TF401349: An unexpected error has occurred, please verify your request and try again.
[Error   @14.03.01.695] 25 total work item(s) failed.
[Error   @14.03.01.699]    25 work item(s) failed for this reason: CriticalError.

Here is the first batch request for updating the Task (280) to Requirement (290) work item child-parent linkage as output in the log file:

WIT BATCH REQUEST 1:

METHOD: PATCH
URI: /_apis/wit/workItems/282?bypassRules=True&suppressNotifications=True&api-version=3.0

BODY:
[{"op":1,"Path":"/relations/0","From":null,"Value":null},{"op":0,"Path":"/relations/-","From":null,"Value":{"rel":"AttachedFile","url":"http://hv-demo-tfs:8080/tfs/DefaultCollection/_apis/wit/attachments/9780e831-76c7-479f-bc99-f89dedde4366","attributes":{"name":"WorkItemHistory267.json"}}},{"op":0,"Path":"/fields/System.Tags","From":null,"Value":"20180303-1800"},{"op":0,"Path":"/relations/-","From":null,"Value":{"rel":"System.LinkTypes.Hierarchy-Reverse","url":"http://hv-demo-tfs:8080/tfs/DefaultCollection/_apis/wit/workItems/290","attributes":{"comment":""}}},{"op":0,"Path":"/relations/-","From":null,"Value":{"rel":"Hyperlink","url":"http://hv-demo-tfs:8080/tfs/DefaultCollection/_apis/wit/workItems/267","attributes":{"id":1018,"comment":"2;attachments;revision history attachments;target post move tags;work item links"}}}]

So I took the information above and fired it off in Postman, resulting in the following:

{
    "$id": "1",
    "innerException": null,
    "message": "TF401349: An unexpected error has occurred, please verify your request and try again.",
    "typeName": "Microsoft.TeamFoundation.WorkItemTracking.Web.Common.UnexpectedException, Microsoft.TeamFoundation.WorkItemTracking.Web",
    "typeKey": "UnexpectedException",
    "errorCode": 0,
    "eventId": 3000
}

Unable to read work item types on the target

I have task, move WI from the one tfs collection to another one. Both tfs collections are located on my TFS 2018 update 2.
So, during migration process I have got this error

“Common.Validation.ValidationException: Unable to read work item types on the target —> Common.RetryPermanentException: Permanent error for b98d5e9f-6d67-41f9-bb6d-01bb68d31d5c, not retrying —> Microsoft.VisualStudio.Services.WebApi.VssServiceResponseException: Page not found.”

Could you please help me with this

support GitHub

💡 (couldn't figure out where else to put this request)

I would like to be able migrate GitHub project to VSTS but so far I can only do the repository part and not issues. Can you expand this project to support that or is there any other repository exists that you may point me?

Please add support for remote links.

Hey Sean, it would be great if the WiMigrator tool could support the new remote link type for ADO:

https://docs.microsoft.com/en-us/azure/devops/boards/backlogs/add-link?view=azure-devops&tabs=browser#link-to-a-remote-work-item

https://docs.microsoft.com/en-us/azure/devops/boards/queries/link-type-reference?view=azure-devops#remote-work-link-types

Thanks! Appreciate the help and for making this tool.

Validation succeeds but migration fails with "Unexpected Error"

As in the subject line. Tried bumping log level up to Debug then Trace, nothing additional found.
the HTTP response is error code 404 with a sample of the batch output as follows:

WIT BATCH REQUEST 1:

METHOD: PATCH
URI: /target-cmmi/_apis/wit/workItems/$Test Suite?bypassRules=True&suppressNotifications=True&api-version=4.0

BODY:
[{"op":0,"Path":"/fields/System.Id","From":null,"Value":278},{"op":0,"Path":"/fields/System.AreaPath","From":null,"Value":"target-cmmi"},{"op":0,"Path":"/fields/System.TeamProject","From":null,"Value":"target-cmmi"},{"op":0,"Path":"/fields/System.NodeName","From":null,"Value":"source-cmmi"},{"op":0,"Path":"/fields/System.Rev","From":null,"Value":1},{"op":0,"Path":"/fields/System.AuthorizedDate","From":null,"Value":"2018-03-03T14:54:26.403Z"},{"op":0,"Path":"/fields/System.RevisedDate","From":null,"Value":"9999-01-01T00:00:00Z"},{"op":0,"Path":"/fields/System.IterationPath","From":null,"Value":"target-cmmi"},{"op":0,"Path":"/fields/System.WorkItemType","From":null,"Value":"Test Suite"},{"op":0,"Path":"/fields/System.State","From":null,"Value":"In Progress"},{"op":0,"Path":"/fields/System.Reason","From":null,"Value":"New test suite"},{"op":0,"Path":"/fields/System.AssignedTo","From":null,"Value":"TFSSetup <CONTOSO\\TFSSetup>"},{"op":0,"Path":"/fields/System.CreatedDate","From":null,"Value":"2018-03-03T14:54:26.403Z"},{"op":0,"Path":"/fields/System.CreatedBy","From":null,"Value":"TFSSetup <CONTOSO\\TFSSetup>"},{"op":0,"Path":"/fields/System.ChangedDate","From":null,"Value":"2018-03-03T14:54:26.403Z"},{"op":0,"Path":"/fields/System.ChangedBy","From":null,"Value":"TFSSetup <CONTOSO\\TFSSetup>"},{"op":0,"Path":"/fields/System.AuthorizedAs","From":null,"Value":"TFSSetup <CONTOSO\\TFSSetup>"},{"op":0,"Path":"/fields/System.Watermark","From":null,"Value":692},{"op":0,"Path":"/fields/System.Title","From":null,"Value":"260 : story 2.1"},{"op":0,"Path":"/fields/Microsoft.VSTS.TCM.TestSuiteTypeId","From":null,"Value":3},{"op":0,"Path":"/fields/Microsoft.VSTS.TCM.TestSuiteType","From":null,"Value":"Requirement Based"},{"op":0,"Path":"/fields/Microsoft.VSTS.TCM.TestSuiteAudit","From":null,"Value":"Added test suite to parent: 273"},{"op":0,"Path":"/id","From":null,"Value":-1},{"op":0,"Path":"/relations/-","From":null,"Value":{"rel":"Hyperlink","url":"http://hv-demo-tfs:8080/tfs/DefaultCollection/_apis/wit/workItems/278","attributes":{"comment":"1"}}}]

RESPONSE CODE:
404

RESPONSE BODY:

work item revision issue

HI Team,

I recently started using this application and thanks for those who created, I just added some enhancements, i will provide to all after my thorough review.

I just met with some work item migration issue and please let me know, if anyone face with same issue or have any solution.

issue:

1. One work item created as - Non Functional Requirement,
2. then changed type into - Change Request,
3. then Changed type into - Bug.
4. The work item operation continues as Bug

The work item is - Bug now.

Now I am trying to move this work item from one VSTS organization to another using below mapping,

Bug -> Bug
Non Functional Requirement ->Story,
Functional Requirement ->Story,
Change Request ->Story,

Started migrating this Bug to another organization with revision history, and at the end I came to know that, bug migrated as a "Story", not as "Bug"

I went with source code review while putting the break points and came to know that, no exceptions are throwing, but the wok item changing from "Change request -> Bug" in revision is not happening.

Anyone can you able to help us to resolve or enhance the comedy.

Regards,
Mebin Thomas

The tool doesn't migrate the shared step in the steps

While the tool migrate the linked shared steps to the test cases, it doesn't migrate it as a step in the steps for the test cases

Unexpected error, critical error, on work item linking

This problem was described in a closed issue but was determined as an error due to version of TFS, but I get the same error using Azure Devops.
After migration phase 1 creates the new work items, it attempts to link them and fails with a rather cryptic TF401349 error. Brief search on the TFS error code turned up very little of value. Here are some of the error lines from the log output:

[Error   @10.09.43.847] Exception during batch 45: TF401349: An unexpected error has occurred, please verify your request and try again.
[Error   @10.09.43.848] Exception during batch 46: TF401349: An unexpected error has occurred, please verify your request and try again.
[Error   @10.09.43.848] Exception during batch 47: TF401349: An unexpected error has occurred, please verify your request and try again.
[Error   @10.09.43.848] 9497 total work item(s) failed.
[Error   @10.09.43.848]    9497 work item(s) failed for this reason: CriticalError.

Here is the first batch request for updating work item child-parent linkage as output in the log file:

WIT BATCH REQUEST 1:

METHOD: PATCH
URI: /_apis/wit/workItems/49774?bypassRules=True&suppressNotifications=True&api-version=4.0

BODY:
[{"op":1,"Path":"/relations/0","From":null,"Value":null},{"op":0,"Path":"/relations/-","From":null,"Value":{"Rel":"AttachedFile","Url":"https://dev.azure.com/FKDayli/_apis/wit/attachments/6b044504-9e97-4477-ae5c-b6c534a1a886","Attributes":{"name":"WorkItemHistory-112880-0.json","resourceSize":10681,"comment":"Update range from 0 to 6"}}},{"op":0,"Path":"/relations/-","From":null,"Value":{"rel":"System.LinkTypes.Hierarchy-Reverse","url":"https://dev.azure.com/FKDayli/_apis/wit/workItems/49623","attributes":{"comment":""}}},{"op":0,"Path":"/relations/-","From":null,"Value":{"rel":"Hyperlink","url":"https://pentia.visualstudio.com/_apis/wit/workItems/112880","attributes":{"id":1758869,"comment":"6;attachments;git commit links;revision history attachments;work item links"}}}]

Images in description field are not strictly copied, but linked to old

I did a migration using the tool and thought everything went well in the end.
After deleting the old project I found that all the images that were in the description field of the work items where gone.

It seems the images in the target projects description was simply linking to the old projects images, so when it was deleted so were their sources.

I realize the fault is my own for not realizing the images were simply linking back to their old ones. But it might be a neat enhancement if images in the description field were re-hosted in the new project.