Giter Site home page Giter Site logo

False Alarms too high! about bellwether HOT 4 OPEN

rahlk avatar rahlk commented on July 19, 2024
False Alarms too high!

from bellwether.

Comments (4)

timm avatar timm commented on July 19, 2024

dude: why is this not i a repo i can participate in?

any why not column justified

cat data | column -s, -t

JDT
name      Pd     Pf     G
lucene     0.88   0.71   0.39
xerces     0.75   0.57   0.32
log4j      0.83   0.44   0.29
xalan      0.59   0.34   0.22
poi        0.49   0.18   0.13
camel      0.50   0.10   0.09
velocity   0.38   0.11   0.08
ant        0.09   0.02   0.02
ivy        0.09   0.01   0.01
jedit      0.03   0.00   0.00
PDE
name      Pd     Pf     G
velocity   0.93   0.99   0.48
lucene     0.60   0.36   0.23
log4j      0.62   0.29   0.20
xerces     0.30   0.45   0.18
poi        0.32   0.21   0.12
ivy        0.27   0.09   0.07
ant        0.14   0.09   0.06
camel      0.10   0.10   0.05
xalan      0.11   0.08   0.05
jedit      0.00   0.00   0.00
EQ
name      Pd     Pf     G
velocity   0.85   0.64   0.37
log4j      0.85   0.54   0.33
lucene     0.88   0.48   0.31
poi        0.87   0.39   0.27
xerces     0.60   0.43   0.25
xalan      0.80   0.34   0.24
camel      0.39   0.11   0.09
ant        0.09   0.04   0.03
jedit      0.08   0.01   0.00
ivy        0.05   0.00   0.00
LC
name      Pd     Pf     G
lucene     1.00   0.94   0.49
log4j      0.95   0.89   0.46
velocity   0.59   0.53   0.28
xerces     0.55   0.28   0.18
xalan      0.50   0.29   0.18
poi        0.52   0.16   0.12
camel      0.19   0.07   0.05
ivy        0.17   0.03   0.03
ant        0.03   0.00   0.00
jedit      0.00   0.00   0.00
ML
name      Pd     Pf     G
log4j      0.71   0.53   0.30
lucene     0.70   0.44   0.27
velocity   0.69   0.40   0.25
xalan      0.56   0.32   0.20
xerces     0.30   0.35   0.16
ivy        0.13   0.07   0.05
poi        0.13   0.05   0.03
camel      0.04   0.01   0.00
ant        0.03   0.00   0.00
jedit      0.00   0.00   0.00
Apache
name      Pd     Pf     G
JDT        0.70   0.23   0.17
PDE        0.40   0.19   0.13
EQ         0.52   0.15   0.11
LC         0.41   0.16   0.11
ML         0.27   0.09   0.07
Safe
name      Pd     Pf     G
EQ         0.86   0.76   0.41
PDE        0.68   0.41   0.26
LC         0.73   0.29   0.21
JDT        0.68   0.29   0.21
ML         0.55   0.21   0.15
Zxing
name      Pd     Pf     G
EQ         0.69   0.60   0.32
JDT        0.58   0.39   0.23
PDE        0.32   0.35   0.17
LC         0.42   0.22   0.15
ML         0.24   0.11   0.08
Apache
name      Pd     Pf     G
lucene     0.96   0.77   0.43
log4j      0.95   0.75   0.42
poi        0.93   0.65   0.38
xalan      0.78   0.42   0.27
xerces     0.70   0.44   0.27
velocity   0.54   0.43   0.24
camel      0.27   0.08   0.06
ivy        0.10   0.02   0.02
ant        0.05   0.02   0.01
jedit      0.02   0.00   0.00
Safe
name      Pd     Pf     G
log4j      0.91   0.82   0.43
xalan      0.91   0.68   0.39
lucene     0.86   0.56   0.34
velocity   0.59   0.76   0.33
xerces     0.73   0.53   0.31
camel      0.82   0.44   0.29
poi        0.68   0.38   0.24
ant        0.59   0.26   0.18
ivy        0.55   0.12   0.10
jedit      0.27   0.15   0.10
Zxing
name      Pd     Pf     G
velocity   0.86   0.95   0.45
lucene     0.89   0.72   0.40
xerces     0.78   0.70   0.37
poi        0.81   0.65   0.36
xalan      0.79   0.57   0.33
log4j      0.72   0.48   0.29
camel      0.33   0.16   0.11
ivy        0.13   0.10   0.06
jedit      0.14   0.05   0.03
ant        0.10   0.05   0.03
Apache
name      Pd     Pf     G
jm         0.80   0.53   0.32
kc         0.70   0.32   0.22
cm         0.31   0.05   0.04
mc         0.02   0.01   0.01
mw         0.00   0.00   0.00
Safe
name      Pd     Pf     G
jm         0.86   0.41   0.28
kc         0.32   0.56   0.20
cm         0.45   0.18   0.13
mw         0.14   0.00   0.00
mc         0.14   0.00   0.00
Zxing
name      Pd     Pf     G
jm         0.64   0.44   0.26
kc         0.41   0.33   0.18
cm         0.08   0.03   0.02
mc         0.03   0.01   0.01
mw         0.00   0.00   0.00
JDT
name      Pd     Pf     G
jm         0.54   0.26   0.17
kc         0.05   0.02   0.01
cm         0.01   0.01   0.00
mw         0.00   0.00   0.00
mc         0.00   0.00   0.00
PDE
name      Pd     Pf     G
jm         0.45   0.16   0.12
kc         0.07   0.05   0.03
cm         0.05   0.02   0.01
mw         0.00   0.00   0.00
mc         0.00   0.00   0.00
EQ
name      Pd     Pf     G
jm         0.67   0.28   0.20
cm         0.13   0.03   0.02
kc         0.10   0.01   0.00
mw         0.00   0.00   0.00
mc         0.05   0.00   0.00
LC
name      Pd     Pf     G
kc         0.80   0.98   0.44
jm         0.69   0.45   0.27
cm         0.16   0.03   0.03
mw         0.02   0.00   0.00
mc         0.02   0.00   0.00
ML
name      Pd     Pf     G
jm         0.26   0.18   0.11
kc         0.26   0.08   0.06
cm         0.04   0.04   0.02
mw         0.00   0.00   0.00
mc         0.00   0.00   0.00
JDT
name      Pd     Pf     G
Apache     0.23   0.05   0.04
Safe       0.23   0.05   0.04
Zxing      0.07   0.06   0.03
PDE
name      Pd     Pf     G
Apache     0.06   0.07   0.03
Safe       0.05   0.06   0.03
Zxing      0.00   0.00   0.00
EQ
name      Pd     Pf     G
Safe       0.48   0.24   0.16
Apache     0.35   0.17   0.11
Zxing      0.04   0.01   0.01
LC
name      Pd     Pf     G
Zxing      0.22   0.10   0.07
Apache     0.05   0.09   0.03
Safe       0.19   0.02   0.02
ML
name      Pd     Pf     G
Apache     0.13   0.19   0.08
Safe       0.23   0.09   0.06
Zxing      0.07   0.07   0.03
ant
name      Pd     Pf     G
JDT        0.81   0.49   0.31
EQ         0.54   0.24   0.17
LC         0.26   0.07   0.05
PDE        0.08   0.05   0.03
ML         0.14   0.03   0.02
xerces
name      Pd     Pf     G
EQ         0.37   0.21   0.14
PDE        0.22   0.21   0.11
JDT        0.19   0.19   0.10
LC         0.22   0.10   0.07
ML         0.03   0.01   0.00
xalan
name      Pd     Pf     G
PDE        0.34   0.26   0.15
EQ         0.32   0.26   0.14
JDT        0.11   0.07   0.04
LC         0.12   0.05   0.04
ML         0.04   0.02   0.01
camel
name      Pd     Pf     G
PDE        0.27   0.20   0.12
LC         0.28   0.13   0.09
EQ         0.18   0.15   0.08
ML         0.10   0.06   0.04
JDT        0.09   0.03   0.02
log4j
name      Pd     Pf     G
JDT        0.75   0.66   0.35
PDE        0.54   0.69   0.30
EQ         0.57   0.48   0.26
ML         0.48   0.33   0.20
LC         0.25   0.03   0.02
lucene
name      Pd     Pf     G
PDE        0.52   0.51   0.26
EQ         0.36   0.16   0.11
LC         0.23   0.10   0.07
JDT        0.09   0.03   0.02
ML         0.05   0.00   0.00
jedit
name      Pd     Pf     G
PDE        0.60   0.44   0.25
EQ         0.59   0.34   0.22
LC         0.20   0.06   0.05
JDT        0.20   0.06   0.04
ML         0.01   0.01   0.01
poi
name      Pd     Pf     G
EQ         0.91   0.76   0.42
PDE        0.17   0.18   0.09
JDT        0.09   0.03   0.02
LC         0.03   0.00   0.00
ML         0.01   0.00   0.00
velocity
name      Pd     Pf     G
LC         0.28   0.15   0.10
JDT        0.23   0.15   0.09
EQ         0.19   0.10   0.06
PDE        0.12   0.07   0.05
ML         0.07   0.03   0.02
ivy
name      Pd     Pf     G
ML         0.40   0.25   0.16
PDE        0.28   0.20   0.12
JDT        0.40   0.16   0.11
EQ         0.39   0.14   0.11
LC         0.22   0.05   0.04
ant
name      Pd     Pf     G
kc         0.10   0.05   0.03
jm         0.02   0.00   0.00
cm         0.01   0.00   0.00
mw         0.00   0.00   0.00
mc         0.00   0.00   0.00
xerces
name      Pd     Pf     G
jm         0.39   0.29   0.17
kc         0.30   0.18   0.11
cm         0.05   0.01   0.00
mw         0.01   0.01   0.00
mc         0.01   0.00   0.00
xalan
name      Pd     Pf     G
jm         0.55   0.34   0.21
kc         0.07   0.03   0.02
cm         0.02   0.01   0.01
mc         0.01   0.01   0.00
mw         0.00   0.00   0.00
camel
name      Pd     Pf     G
kc         0.56   0.46   0.25
jm         0.40   0.30   0.17
cm         0.00   0.00   0.00
mw         0.00   0.00   0.00
mc         0.00   0.00   0.00
log4j
name      Pd     Pf     G
jm         0.78   0.69   0.37
kc         0.35   0.31   0.17
cm         0.31   0.22   0.13
mw         0.00   0.00   0.00
mc         0.00   0.00   0.00
lucene
name      Pd     Pf     G
jm         0.77   0.70   0.37
kc         0.04   0.01   0.01
cm         0.02   0.00   0.00
mw         0.00   0.00   0.00
mc         0.00   0.00   0.00
jedit
name      Pd     Pf     G
jm         0.38   0.37   0.19
kc         0.05   0.01   0.01
mc         0.04   0.00   0.00
cm         0.00   0.00   0.00
mw         0.00   0.00   0.00
poi
name      Pd     Pf     G
kc         0.76   0.53   0.31
jm         0.51   0.27   0.18
mw         0.00   0.00   0.00
cm         0.00   0.00   0.00
mc         0.00   0.00   0.00
velocity
name      Pd     Pf     G
kc         0.34   0.31   0.16
jm         0.10   0.09   0.05
cm         0.05   0.06   0.03
mw         0.01   0.00   0.00
mc         0.00   0.00   0.00
ivy
name      Pd     Pf     G
jm         0.22   0.07   0.05
cm         0.02   0.00   0.00
kc         0.00   0.00   0.00
mw         0.00   0.00   0.00
mc         0.01   0.00   0.00
ant
name      Pd     Pf     G
Safe       0.21   0.21   0.11
Zxing      0.03   0.11   0.02
Apache     0.01   0.01   0.00
xerces
name      Pd     Pf     G
Zxing      0.41   0.54   0.23
Apache     0.24   0.16   0.10
Safe       0.17   0.11   0.07
xalan
name      Pd     Pf     G
Safe       0.14   0.15   0.07
Zxing      0.15   0.12   0.07
Apache     0.05   0.03   0.02
camel
name      Pd     Pf     G
Zxing      0.28   0.25   0.13
Safe       0.25   0.22   0.12
Apache     0.29   0.17   0.11
log4j
name      Pd     Pf     G
Zxing      0.14   0.10   0.06
Apache     0.27   0.03   0.03
Safe       0.01   0.00   0.00
lucene
name      Pd     Pf     G
Zxing      0.32   0.21   0.13
Safe       0.32   0.20   0.12
Apache     0.30   0.14   0.10
jedit
name      Pd     Pf     G
Zxing      0.19   0.18   0.09
Apache     0.04   0.04   0.02
Safe       0.06   0.01   0.01
poi
name      Pd     Pf     G
Apache     0.03   0.02   0.01
Safe       0.02   0.02   0.01
Zxing      0.00   0.02   0.00
velocity
name      Pd     Pf     G
Zxing      0.25   0.29   0.13
Safe       0.17   0.13   0.07
Apache     0.06   0.06   0.03
ivy
name      Pd     Pf     G
Zxing      0.05   0.11   0.04
Apache     0.06   0.02   0.01
Safe       0.06   0.01   0.01
kc
name      Pd     Pf     G
PDE        0.67   0.53   0.29
JDT        0.56   0.38   0.23
EQ         0.53   0.34   0.21
LC         0.56   0.32   0.21
ML         0.25   0.30   0.14
jm
name      Pd     Pf     G
EQ         0.16   0.06   0.05
PDE        0.08   0.06   0.04
ML         0.02   0.00   0.00
LC         0.02   0.00   0.00
JDT        0.02   0.00   0.00
mw
name      Pd     Pf     G
EQ         0.89   0.78   0.41
PDE        0.85   0.75   0.40
JDT        0.74   0.50   0.30
LC         0.74   0.43   0.27
ML         0.44   0.21   0.14
cm
name      Pd     Pf     G
JDT        0.52   0.35   0.21
EQ         0.62   0.30   0.20
LC         0.48   0.22   0.15
PDE        0.33   0.21   0.13
ML         0.24   0.08   0.06
mc
name      Pd     Pf     G
JDT        0.93   0.84   0.44
LC         0.89   0.68   0.39
ML         0.70   0.32   0.22
EQ         0.63   0.32   0.21
PDE        0.33   0.24   0.14
kc
name      Pd     Pf     G
log4j      0.78   0.88   0.41
lucene     0.86   0.61   0.36
poi        0.78   0.64   0.35
camel      0.78   0.59   0.33
xalan      0.72   0.61   0.33
xerces     0.58   0.54   0.28
velocity   0.39   0.44   0.21
ant        0.22   0.07   0.06
ivy        0.14   0.04   0.03
jedit      0.08   0.03   0.02
jm
name      Pd     Pf     G
velocity   0.91   0.85   0.44
xerces     0.50   0.61   0.28
log4j      0.66   0.45   0.27
xalan      0.42   0.41   0.21
lucene     0.24   0.10   0.07
camel      0.13   0.03   0.02
poi        0.08   0.02   0.02
ivy        0.03   0.01   0.01
ant        0.01   0.00   0.00
jedit      0.00   0.00   0.00
mw
name      Pd     Pf     G
xalan      1.00   0.98   0.49
poi        0.96   0.87   0.46
log4j      1.00   0.84   0.46
velocity   1.00   0.83   0.45
lucene     0.85   0.68   0.38
camel      0.81   0.59   0.34
ant        0.59   0.64   0.31
xerces     0.78   0.47   0.29
ivy        0.67   0.35   0.23
jedit      0.63   0.28   0.19
cm
name      Pd     Pf     G
xalan      0.98   0.91   0.47
velocity   0.93   0.89   0.45
poi        0.98   0.84   0.45
lucene     0.95   0.78   0.43
log4j      0.90   0.69   0.39
xerces     0.60   0.71   0.32
camel      0.50   0.29   0.18
ivy        0.45   0.21   0.14
ant        0.24   0.10   0.07
jedit      0.14   0.08   0.05
mc
name      Pd     Pf     G
log4j      1.00   0.99   0.50
lucene     1.00   0.96   0.49
poi        1.00   0.92   0.48
xalan      0.96   0.81   0.44
xerces     0.93   0.79   0.43
velocity   0.63   0.60   0.31
camel      0.56   0.38   0.23
jedit      0.56   0.33   0.21
ivy        0.56   0.31   0.20
ant        0.48   0.12   0.09
kc
name      Pd     Pf     G
Apache     0.72   0.57   0.32
Zxing      0.22   0.37   0.14
Safe       0.17   0.06   0.05
jm
name      Pd     Pf     G
Apache     0.06   0.03   0.02
Zxing      0.06   0.02   0.01
Safe       0.02   0.01   0.01
mw
name      Pd     Pf     G
Safe       0.59   0.36   0.22
Zxing      0.33   0.32   0.16
Apache     0.22   0.04   0.04
cm
name      Pd     Pf     G
Apache     0.52   0.36   0.21
Safe       0.17   0.09   0.06
Zxing      0.00   0.00   0.00
mc
name      Pd     Pf     G
Safe       0.67   0.30   0.21
Apache     0.41   0.18   0.13
Zxing      0.15   0.12   0.07

from bellwether.

timm avatar timm commented on July 19, 2024

btw... does this mean the bellwether paper is crap? what are the pd,pf reports there?

from bellwether.

rahlk avatar rahlk commented on July 19, 2024
  • I think this has to do with SMOTE in HDP (Adding/removing columns randomly impacts the datasets). Running experiments to confirm/reject this hypothesis.
  • As for Bellwether = BS — I don't think so. I'm reruning those experiments for pd, pfs to verify. I'll post updates here.

Also, I have added you as a collaborator.

from bellwether.

timm avatar timm commented on July 19, 2024

cool

from bellwether.

Related Issues (11)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.