How did do you obtain the precision, accuracy, etc for Pairwise LDA?

[suderoy]

How did do you obtain the precision, accuracy, etc for Pairwise LDA?

[suderoy]

Please follow the "Baseline Method" section in the paper for exact definition and formula of Pairwise-LDA.

Definition of precision/recall are standard as in https://en.wikipedia.org/wiki/Precision_and_recall

[mhoangvslev]

I am stuck at this step:

#Exponentiate and normaliezed
for ct in range(beta_n_ct):
    beta[:, ct] = np.exp(beta[:, ct])/max(beta[:, ct])
    #beta[:, ct] = softmax(beta[:, ct])

for cs in range(beta_n_cs):
    for ct in range(beta_n_ct):
        k = f"{vocab[cs]},{vocab[ct]}"
        v = beta[cs, :].T @ eta @ beta[:, ct]
        print(f"({k}): {v}")
        score[k] = v

and the result looks like this:

(swine flu leading,disease impact society): -9.067590602136704e-87
(swine flu leading,rubber ball professor): -8.268603955448518e-46
(swine flu leading,polymerizations catalyzed free): -8.268603955448518e-46
(swine flu leading,swine flu leading): -9.067590602136704e-87

I can't use the threshold 0.5 to distinguish classes. Do you have any suggestion?

Suppose that I have the classes, I can just use the number of Positive (or Negative) classes / topK? How do I proceed?

[suderoy]

you need to normalize the values.

…

On Thu, Jan 14, 2021 at 5:22 PM Minh-Hoang DANG ***@***.***> wrote: I am stuck at this step: #Exponentiate and normaliezed for ct in range(beta_n_ct): beta[:, ct] = np.exp(beta[:, ct])/max(beta[:, ct]) #beta[:, ct] = softmax(beta[:, ct]) for cs in range(beta_n_cs): for ct in range(beta_n_ct): k = f"{vocab[cs]},{vocab[ct]}" v = beta[cs, :].T @ eta @ beta[:, ct] print(f"({k}): {v}") score[k] = v and the result looks like this: (swine flu leading,disease impact society): -9.067590602136704e-87 (swine flu leading,rubber ball professor): -8.268603955448518e-46 (swine flu leading,polymerizations catalyzed free): -8.268603955448518e-46 (swine flu leading,swine flu leading): -9.067590602136704e-87 I can't use the threshold 0.5 to distinguish classes. Do you have any suggestion? Suppose that I have the classes, I can just use the number of Positive (or Negative) classes / topK? How do I proceed? — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub <#6 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB7O5NZPMM3UPGAC3GEAQ6DSZ6KE7ANCNFSM4WCK7EWQ> .

-- Sudeshna Roy

[mhoangvslev]

which one? score[k] = v or beta or eta or both? Also, did I get this part right? (v = beta[cs, :].T @ eta @ beta[:, ct])

final.beta example:

-100.0000000000 -408.7050370236 -409.6351143006 -100.0000000000 -100.0000000000 -406.7467220222 -409.2920602431 -100.0000000000 -100.0000000000 -406.1868127315 -405.3547537916 -100.0000000000 -100.0000000000 -406.2035637767 -100.0000000000 -100.0000000000 -405.3715661278 -404.9402052329 -100.0000000000 -100.0000000000 -406.3623611021 -406.8360580116 -100.0000000000 -100.0000000000 -408.4683356327 -409.0364734684 -100.0000000000 -412.0738675058 -411.3081752542 -403.8635220151 -406.8025649053 -407.7871455696 -410.0841176625 -407.8713929676 -100.0000000000 -100.0000000000 -403.9410011342 -403.5545719647 -100.0000000000 -100.0000000000 -409.4411510096 -408.0252104539 -100.0000000000 -405.9784221236 -413.9470114693 -409.9124939402 -100.0000000000 -409.8844661788 -405.5522822216 -405.6279164250 -100.0000000000 -398.8833686187 -396.8032394491 -409.1952658656 -405.4179719063 -100.0000000000 -100.0000000000 -407.4994967708
...

final.eta:

 -20.2601991200 -16.7163315013 -20.0531020540 -20.0560904910 -20.0749574297 -5.6176003101 -20.1416540267 -16.0983732590 -14.3091306179 -20.0448913131 -20.0762214530 -20.0011341731 -19.9902476909 -20.2232805604 -20.1445525817 -20.0345875666 -19.0888845418 -19.9637615302 -18.5845158864 -19.9953873629 -16.3175007916 -15.6007154075 -5.6698288753 -20.1132851488 -20.0649505210 -20.1071892557 -19.9936755746 -19.9834904842 -20.0970950907 -15.7540496836 -20.0526120087 -20.0127836933 -19.9453379171 -19.9782117655 -20.0793742185 -19.9914881946 -20.0509856589 -20.1711639298 -19.9999809313 -20.0024620855 -20.0110466124 -20.0683708857 -19.9125406500 -19.9655397625 -14.9717923503 -19.9953119600 -5.8942895553 -16.5250983261 -20.0486258756 -13.9586031026 -19.9261256706 -20.0371649084 -14.9248751907 -20.0020771582 -19.9685397681 -20.1368620982 -20.0744943355 -20.0358019846 -19.9947220393 -19.9478529765 -20.2196108944 -20.0036554027 -19.9476872881 -20.1792339316 -20.0979932765 -20.1623802390 -20.4537626251 -20.0857257902 -19.9660737795 -20.1119491764 -19.9803700979 -20.1121208979 -20.0687069950 -20.0172156188 -17.0973889482 -20.0901011541 -20.0299802482 -20.0267073245 -20.0879002913 -19.9959439429 -20.0077530483 -20.0985879479 -20.0515980494 -20.0784422867 -20.0355349837 -20.1839854704 -20.0062455641 -20.0911203766 -20.0187495486 -20.0966086971 -20.0421874271 -15.5179361983 -20.1016796738 -19.9651287187 -20.2239907787 -20.1170018287 -19.9918675020 -18.5140795898 -20.0028993216 -17.2145815757
...