Bot will repeatedly set trunk as affected if we fix an issue via a pref-flip and represent that as `firefoxN: disabled`

[dholbert]

Bot will repeatedly set trunk as affected if we fix an issue via a pref-flip and represent that as `firefoxN: disabled`

[dholbert]

(You're correct that it does show up on the release-manager regression-triage-list, and for now would be cleaned up by that team manually. The reason I noticed this issue in the first place because I'm part of the release manager regression-triage meeting for the 128 release cycle, and I happened to notice the bug-in-question here in an early on-my-own skim through the bugs-to-be-triaged list. In my (limited) experience with the release-manager regression-triage-meetings so far, I'm noticing the meeting is quite time-constrained and doesn't always make it through the full bug list -- so if we can avoid making bugs unnecessarily show up in the lists there, it leaves more time for the folks in that meeting to get further in the list to other bugs that may actually need attention.)

[dholbert]

(2) having the bot detect this special situation and notify someone (assignee or triage owner) with a comment like the following (hastily written, please don't use this exact text :))
"Bug has been closed, but no version got a status fixed, so the assumption is that future Firefox versions should automatically be marked as affected. This is probably a mistake - should some Firefox version have a fixed status?"

Or maybe more generally:
(3) have the bot detect and nag about a broader situation that includes this scenario, i.e. bugs that are marked as affecting some version (affected/wontfix/fix-optional/disabled) that were closed as fixed but never had any version flagged as fixed/verified. Wording for the nag might be along these lines:

"Bug is flagged as affecting some version, and was closed as FIXED, but no versions were set to status fixed (or verified). Could you set the status flag to fixed for the version where the fix landed, so we can reason about at-which-point future versions can also be implicitly considered fixed?"

[suhaibmujahid]

See also: #1535

[suhaibmujahid]

In such a scenario, the bug should probably show up for the release managers during their regression triage, and they can flag it as fixed when appropriate.

@rvandermeulen what do you think?

[dholbert]

Right, but that won't happen until after the bot has intervened to (incorrectly) mark e.g. status-firefox128 as affected, which then requires folks to notice, figure out that 128 is not in fact affected, figure out why the bot did that, and then clean up after it by resetting that flag and then resetting some earlier flag to the state that the bot was expecting.

That feels sub-optimal.

[suhaibmujahid]

That feels sub-optimal.

I totally agree. I want first to confirm with release managers that ignoring setting the flag for such cases would not cause an unexpected impact on their workflows/queries.

[rvandermeulen]

Discussed it with the team and TBH we don't have very strong opinions on this. We can see situations where a false positive could occur with or without this change. But overall, maybe it doesn't make sense to make statusN+1 flag changes to bugs that are set to RESOLVED/VERIFIED? I can see the use case for statusN-1 around uplifts, but what's the case higher release versions?

[dholbert]

FWIW: I'm not necessarily suggesting that the bot not-set-the-flag in this situation. That was my suggestion (1) in the initial comment here, and perhaps it's better than the current state of things; but if possible, I'd lean more towards automatically catching the situation earlier (around when the bug is closed) and poking the assignee or triage-owner to update the status flags at that point, if appropriate, per my suggested option (3) above ( #2403 (comment) ), if the bot sees that the bug is closed and yet it can predict that it's going to mark future versions as affected.