polarization fidelity is not a valid comparator

[sri-international]

polarization fidelity is not a valid comparator

[nonhermitian]

Actually the above is valid save for the case where the ideal distribution is also the uniform distribution for which the denominator of Eq.(2) goes to zero. I am not sure how to solve that pathological case.

[rtvuser1]

Yes, that is one of the issues that the team wrestled with (the divide by zero case). We settled on polarization fidelity (as defined in the repository) as a good compromise that addresses the requirements of the current suite of benchmark programs.

…

On Sun, Feb 13, 2022 at 7:41 AM Paul Nation ***@***.***> wrote: Actually the above is valid save for the case where the ideal distribution is also the uniform distribution for which the denominator of Eq.(2) goes to zero. I am not sure how to solve that pathological case. — Reply to this email directly, view it on GitHub <#47 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADIPLWUOJKEONW2BWXHDGJDU27GJXANCNFSM5OJENK2Q> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you are subscribed to this thread.Message ID: <SRI-International/QC-App-Oriented-Benchmarks/issues/47/1038202471@ github.com>

[nonhermitian]

Actually I messed up the math above, maximizing the wrong thing. The worst case is given by an experiment that returns a single bit-string, but the actual answer is a uniform distribution over all other bit-strings. In this case one gets an overlap with the uniform distribution of 1-1/2**N so the lower bound is:

-(1-1/2**N)/(1-((1-1/2**N))

Which a quick check equals -2^{N}+1. Which is still qubit number dependent.

In short, the more important point is not really the zero division, but that your are equating things with fidelities over different ranges. And the fact that the lower bound is negative, but that is not anywhere in the plots.

[necaisej]

It should be specified in the paper that we are reporting all negative fidelities as 0, for the following reason; forgive me if this is repeating what you already know, but this is a representation of the thought process behind the implementation:

The choice of normalizing to some large-noise limit distribution, in this case the uniform distribution over bitstrings, is intended to clarify the actual ability of a quantum processor to execute its instructions (without tomography) and also its ability to achieve a task better than some easy classical task (random guessing). Polarization fidelity is negative when the Hellinger fidelity of the large-noise limit distribution with the exact distribution is higher than the Hellinger fidelity of the actual outputs with the exact distribution. So "less informative than noise" is conflated with "indistinguishable from noise." It's a fair criticism of the metric that these are not quite the same cases but in reporting whether your QPU can perform the task that an algorithm is designed to do, we are currently choosing to set them both as equal to 0.