BB Issue IDs may not match Github Issue IDs if BB issues were deleted

[jeffwidman]

BB Issue IDs may not match Github Issue IDs if BB issues were deleted

[jaraco]

I encountered this issue today attempting to migrate CherryPy with a missing issue.

[jeffwidman]

This is a legit problem--example: https://bitbucket.org/zzzeek/sqlalchemy/issues/2479

[jeffwidman]

Added a test to catch this in 0dc1824

[jeffwidman]

As noted in #70 (comment), can't just use track the offset here for fix_links() because of the following scenario:

1. issue 1 created
2. issue 2 created
3. issue 3 created, and then issue 1 edited to include a reference to issue 3.
4. issue 2 deleted.

If I just track an offset, when the link in issue 1 to issue 3 is converted, the way the script is currently written means it won't be aware of the offset created by issue 2 being deleted. Could do a mapping, but that feels a bit overblown.

So probably the most pragmatic solution is still to insert blank placeholder issue.

[mithrandi]

I just wanted to point out that it's possible for issues to be non-contiguous even if no issues were deleted; for example, when we originally migrated from Launchpad to BitBucket, we kept the Launchpad bug numbers for everything. Since Launchpad bug numbers are global (a single bug can be associated with multiple projects), our bug numbers start at a high number (#433605), and have lots and lots of holes (latest issue is #1228543).

[jeffwidman]

@mithrandi thanks, I was ignorant of that--definitely affects the proposed solution here. Likely I'll need to instead just make the link converter function optional, so that it can be disabled for these scenarios.

Did Bitbucket support specifying the issue ID when you imported over from Launchpad?

[mithrandi]

It's been a while since we did the migration to BitBucket (and I can't find the script we wrote to do it), but I believe it was done by preparing the data in a suitable format for the Import/Export feature, documented here. I don't think you can create an issue through the API with an arbitrary issue ID, but you can specify it in the import data.

By the looks of things, the GitHub issues import API does not support this, and creating hundreds of thousands of placeholder issues is obviously not a great idea. Perhaps they would add it if we ask nicely? On the other hand, at least in our particular case, we probably don't care too much about keeping the issue IDs, we mainly did it in the LP→BB migration because we could, not because it was important. (If it wasn't clear, we're looking to migrate from BB→GH now).

[mithrandi]

Aha, found it the script: https://bitbucket.org/jonathanj/lp2bb

[jeffwidman]

@mithrandi the GitHub issue import API uses a task queue to process/import tasks, they're not immediately created. So I doubt they'd add support for specifying IDs, as it'd likely require changing their whole infrastructure underlying issue importing.

Given the wide range of holes for issue IDs from Launchpad, sounds like actually best to create a mapping of issue IDs in BB to expected ID in GitHub. This mapping should also account for existing issues in the GH destination repo (#70). Then run the import (and hope that nobody files a new issue during the import process... probably add a note to the docs encouraging folks who really care to start with blank repo, and only migrate the code if the issues migrate successfully, otherwise to redo the issue import).

This will require fetching all issues & issue comments from BB before starting to import anything into GitHub so that they can all be processed in one go once the mapping is finished. Currently comments are fetched as an issue is imported.

Also could make fix_links() controlled by an optional flag on teh command line, defaulting to enabled, but that way people disable if desired. Personally, I think it's extremely convenient to run fix_links() for updating crosslinks between issues.

[mithrandi]

You should probably include the original BB ID somewhere in the migrated issue at least, since this may appear elsewhere (eg. in VCS commits that were migrated).

[thomasjsn]

@jaraco Thanks for your work on filling gaps, work beautifully :)