rahlk / bellwether Goto Github PK
View Code? Open in Web Editor NEWSimpler Transfer Learning (Using "Bellwethers"). ARXIV link: https://arxiv.org/abs/1703.06218
Simpler Transfer Learning (Using "Bellwethers"). ARXIV link: https://arxiv.org/abs/1703.06218
## ant
name Pd Pf G
PDE 1.00 0.99 0.96
JDT 0.96 0.93 0.86
EQ 0.92 0.74 0.79
ML 0.83 0.73 0.72
LC 0.84 0.51 0.59
## xerces
name Pd Pf G
PDE 0.97 0.97 0.96
ML 0.88 0.77 0.83
JDT* 0.62 0.44 0.51
LC 0.53 0.46 0.49
EQ 0.46 0.40 0.45
## xalan
name Pd Pf G
ML 1.00 0.99 0.99
PDE 0.99 0.99 0.97
EQ 0.73 0.61 0.66
JDT 0.60 0.40 0.51
LC 0.32 0.20 0.25
## camel
name Pd Pf G
PDE 0.92 0.84 0.87
ML 0.92 0.86 0.83
JDT 0.86 0.80 0.79
EQ 0.79 0.64 0.71
LC 0.46 0.27 0.35
## log4j
name Pd Pf G
JDT 0.97 0.96 0.95
PDE 0.98 0.95 0.93
EQ 0.91 0.86 0.87
ML 0.90 0.81 0.84
LC 0.79 0.63 0.65
## lucene
name Pd Pf G
ML 0.99 0.94 0.96
PDE 0.96 0.92 0.94
LC 0.76 0.54 0.62
JDT* 0.61 0.46 0.50
EQ 0.60 0.38 0.48
## jedit
name Pd Pf G
PDE 0.95 0.95 0.93
ML 0.97 0.88 0.92
JDT* 0.66 0.39 0.52
LC 0.47 0.20 0.28
EQ 0.34 0.09 0.18
## poi
name Pd Pf G
PDE 1.00 0.97 0.97
ML 0.92 0.73 0.80
JDT 0.90 0.70 0.79
EQ 0.84 0.61 0.66
LC 0.57 0.36 0.49
## velocity
name Pd Pf G
PDE 0.93 0.89 0.90
JDT 0.93 0.89 0.90
ML 0.77 0.73 0.72
EQ 0.76 0.72 0.71
LC 0.27 0.24 0.34
## ivy
name Pd Pf G
PDE 1.00 0.99 0.99
JDT 0.93 0.80 0.86
ML 0.86 0.61 0.75
EQ * 0.74 0.45 0.57
LC * 0.62 0.38 0.49
## Apache
name Pd Pf G
xerc 1.00 1.00 1.00
xala 1.00 1.00 1.00
velo 0.98 0.99 0.97
came 0.96 0.88 0.92
luce 0.98 0.85 0.91
log4 0.97 0.86 0.85
poi 0.92 0.70 0.80
jedi 0.87 0.60 0.69
ant* 0.72 0.46 0.57
ivy* 0.71 0.40 0.51
## Safe
name Pd Pf G
xala 1.00 1.00 1.00
xerc 1.00 1.00 1.00
poi 1.00 0.94 0.96
luce 1.00 0.93 0.96
velo 0.95 1.00 0.94
came 0.95 0.85 0.90
log4 1.00 0.82 0.90
ant 0.91 0.78 0.77
jedi 0.89 0.65 0.73
ivy 0.91 0.53 0.64
## Zxing
name Pd Pf G
xerc 1.00 1.00 1.00
xala 1.00 1.00 1.00
came 1.00 1.00 1.00
log4 1.00 1.00 0.97
velo 0.95 0.98 0.96
luce 0.95 0.81 0.88
poi 0.93 0.82 0.87
ant 0.87 0.82 0.82
jedi 0.86 0.75 0.77
ivy 0.74 0.59 0.65
## kc
name Pd Pf G
Zxin 0.53 0.49 0.51
Apac 0.51 0.47 0.50
Safe 0.31 0.10 0.16
## jm
name Pd Pf G
Apac 0.74 0.78 0.72
Zxin 0.26 0.10 0.18
Safe 0.04 0.01 0.02
## mw
name Pd Pf G
Apac 0.80 0.65 0.68
Zxin 0.65 0.62 0.60
Safe 0.17 0.08 0.12
## cm
name Pd Pf G
Zxin 0.76 0.62 0.70
Apac 0.40 0.28 0.34
Safe 0.17 0.14 0.16
## mc
name Pd Pf G
Zxin 0.90 0.73 0.79
Apac 0.57 0.41 0.49
Safe 0.08 0.04 0.06
## kc
name Pd Pf G
JDT 1.00 0.99 0.99
PDE 1.00 1.00 0.99
LC 0.97 0.95 0.95
ML 0.99 0.93 0.95
EQ 1.00 0.95 0.93
## jm
name Pd Pf G
PDE 0.95 0.91 0.91
ML 0.78 0.65 0.75
JDT 0.24 0.10 0.22
EQ 0.26 0.12 0.18
LC 0.22 0.07 0.11
## mw
name Pd Pf G
EQ 1.00 0.92 0.94
ML 1.00 0.88 0.93
JDT 1.00 0.92 0.93
LC 1.00 0.85 0.92
PDE 0.96 0.86 0.87
## cm
name Pd Pf G
ML 1.00 0.95 0.97
PDE 0.87 0.90 0.83
JDT 0.86 0.63 0.74
LC 0.88 0.62 0.72
EQ 0.79 0.66 0.69
## mc
name Pd Pf G
PDE 1.00 0.99 0.99
ML 1.00 1.00 0.99
JDT 0.98 0.95 0.92
LC 0.89 0.58 0.69
EQ 0.85 0.56 0.68
## JDT
name Pd Pf G
xerc 1.00 1.00 1.00
xala 1.00 1.00 1.00
velo 1.00 1.00 0.99
luce 1.00 0.98 0.98
came 0.98 0.95 0.97
log4 0.97 0.96 0.91
ant 0.92 0.74 0.79
poi* 0.76 0.40 0.56
jedi* 0.67 0.42 0.55
ivy 0.57 0.26 0.38
## PDE
name Pd Pf G
xala 1.00 1.00 1.00
xerc 1.00 1.00 0.99
velo 0.99 0.99 0.98
luce 0.97 0.89 0.93
log4 0.97 0.94 0.91
poi 0.95 0.91 0.90
jedi 0.90 0.74 0.73
ant* 0.68 0.42 0.53
came 0.58 0.23 0.41
ivy 0.34 0.21 0.31
## EQ
name Pd Pf G
xerc 1.00 1.00 1.00
xala 1.00 1.00 1.00
luce 1.00 1.00 0.96
velo 0.97 0.98 0.96
poi 0.99 0.95 0.92
came 0.97 0.83 0.86
log4 0.99 0.84 0.86
jedi 0.68 0.64 0.64
ivy* 0.87 0.45 0.60
ant 0.44 0.20 0.36
## LC
name Pd Pf G
xala 1.00 1.00 1.00
xerc 1.00 1.00 1.00
velo 0.97 0.99 0.98
came 1.00 0.96 0.97
luce 0.98 0.96 0.95
poi 0.93 0.85 0.86
log4 0.93 0.85 0.85
ant 0.84 0.63 0.67
jedi 0.59 0.52 0.56
ivy 0.58 0.33 0.39
## ML
name Pd Pf G
xala 1.00 1.00 1.00
xerc 1.00 1.00 1.00
velo 1.00 1.00 0.99
luce 0.88 0.88 0.88
log4 0.92 0.88 0.84
came 0.86 0.81 0.84
poi 0.87 0.81 0.79
ivy 0.82 0.65 0.64
ant 0.59 0.59 0.56
jedi 0.38 0.27 0.32
## kc
name Pd Pf G
xala 1.00 1.00 1.00
xerc 1.00 1.00 1.00
poi 1.00 0.99 1.00
came 1.00 0.99 0.99
log4 1.00 0.99 0.98
luce 1.00 0.96 0.98
velo 0.97 0.98 0.98
jedi 0.89 0.71 0.78
ant 0.86 0.64 0.72
ivy 0.81 0.61 0.67
## jm
name Pd Pf G
xala 1.00 1.00 1.00
velo 1.00 1.00 1.00
xerc 1.00 1.00 0.99
poi 0.98 0.96 0.94
log4 0.99 0.96 0.94
luce 0.94 0.88 0.81
came 0.88 0.73 0.78
ant 0.66 0.55 0.60
jedi 0.34 0.19 0.32
ivy 0.19 0.06 0.12
## mw
name Pd Pf G
xerc 1.00 1.00 1.00
xala 1.00 1.00 1.00
log4 1.00 0.99 1.00
luce 1.00 0.96 0.98
poi 1.00 0.98 0.97
velo 0.96 0.99 0.95
ant 1.00 0.93 0.95
jedi 0.96 0.89 0.85
ivy 0.93 0.71 0.80
came 0.93 0.74 0.73
## cm
name Pd Pf G
xala 1.00 1.00 1.00
xerc 1.00 1.00 1.00
velo 0.99 0.99 0.98
luce 1.00 0.95 0.97
poi 0.98 0.90 0.92
log4 0.95 0.83 0.88
jedi 0.98 0.81 0.88
came 0.79 0.64 0.76
ivy 0.90 0.67 0.76
ant 0.86 0.74 0.71
## mc
name Pd Pf G
xala 1.00 1.00 1.00
xerc 1.00 1.00 1.00
luce 1.00 0.99 0.98
velo 1.00 0.99 0.98
log4 1.00 1.00 0.96
came 1.00 0.98 0.96
poi 1.00 0.91 0.95
ant 0.95 0.93 0.90
ivy 0.95 0.90 0.87
jedi 0.92 0.65 0.77
## ant
name Pd Pf G
jm 1.00 1.00 1.00
cm * 0.60 0.32 0.39
kc 0.14 0.04 0.09
mw 0.00 0.00 0.00
mc 0.00 0.00 0.00
## xerces
name Pd Pf G
jm 0.99 0.99 0.97
cm 0.32 0.28 0.30
kc 0.15 0.08 0.10
mc 0.09 0.05 0.06
mw 0.01 0.00 0.00
## xalan
name Pd Pf G
jm 1.00 1.00 1.00
kc 0.20 0.19 0.18
cm 0.08 0.05 0.07
mc 0.05 0.02 0.04
mw 0.01 0.01 0.00
## camel
name Pd Pf G
jm 1.00 1.00 0.99
kc 0.58 0.48 0.49
mc 0.12 0.06 0.10
cm 0.06 0.02 0.05
mw 0.02 0.01 0.01
## log4j
name Pd Pf G
jm 1.00 1.00 0.99
cm 0.62 0.54 0.54
kc 0.08 0.05 0.07
mc 0.15 0.02 0.06
mw 0.00 0.00 0.00
## lucene
name Pd Pf G
jm 1.00 1.00 0.99
kc 0.26 0.23 0.25
mc 0.11 0.07 0.10
mw 0.00 0.00 0.00
cm 0.01 0.00 0.00
## jedit
name Pd Pf G
jm 1.00 1.00 1.00
kc 0.45 0.19 0.25
mc 0.07 0.01 0.03
cm 0.02 0.01 0.01
mw 0.01 0.00 0.00
## poi
name Pd Pf G
jm 1.00 0.98 0.98
mc 0.19 0.08 0.11
kc 0.05 0.05 0.08
cm 0.03 0.02 0.02
mw 0.01 0.00 0.00
## velocity
name Pd Pf G
jm 1.00 1.00 1.00
mc 0.07 0.05 0.06
cm 0.06 0.03 0.04
kc 0.01 0.01 0.01
mw 0.00 0.00 0.00
## ivy
name Pd Pf G
jm 1.00 1.00 0.98
cm 0.29 0.12 0.21
kc 0.08 0.12 0.10
mc 0.04 0.01 0.02
mw 0.01 0.00 0.00
## Apache
name Pd Pf G
ML 0.98 0.96 0.95
PDE 0.99 0.93 0.95
JDT 0.98 0.90 0.92
LC 0.85 0.51 0.63
EQ * 0.73 0.42 0.55
## Safe
name Pd Pf G
ML 1.00 1.00 0.98
JDT 1.00 0.96 0.97
PDE 0.93 0.91 0.90
EQ 0.91 0.82 0.87
LC 0.82 0.53 0.66
## Zxing
name Pd Pf G
PDE 1.00 0.99 0.98
ML 0.98 0.98 0.96
JDT 0.83 0.70 0.77
EQ 0.71 0.58 0.62
LC 0.74 0.54 0.62
## Apache
name Pd Pf G
jm 1.00 1.00 1.00
kc * 0.61 0.29 0.36
mc 0.48 0.18 0.24
cm 0.21 0.08 0.11
mw 0.04 0.02 0.00
## Safe
name Pd Pf G
jm 1.00 1.00 0.99
cm * 0.68 0.29 0.41
mc 0.59 0.24 0.32
kc 0.30 0.10 0.00
mw 0.18 0.00 0.00
## Zxing
name Pd Pf G
jm 1.00 1.00 0.99
kc 0.41 0.30 0.36
mc 0.24 0.12 0.17
cm 0.14 0.08 0.10
mw 0.02 0.01 0.01
## JDT
name Pd Pf G
jm 0.98 0.95 0.93
kc 0.28 0.05 0.10
cm 0.11 0.00 0.01
mc 0.04 0.00 0.00
mw 0.00 0.00 0.00
## PDE
name Pd Pf G
jm 1.00 1.00 0.94
kc 0.33 0.09 0.14
mw 0.01 0.00 0.00
cm 0.00 0.00 0.00
mc 0.00 0.00 0.00
## EQ
name Pd Pf G
jm 1.00 1.00 0.99
kc * 0.69 0.36 0.43
cm 0.29 0.07 0.12
mc 0.14 0.02 0.04
mw 0.02 0.00 0.00
## LC
name Pd Pf G
jm 1.00 1.00 1.00
kc 0.80 0.59 0.64
cm 0.08 0.03 0.04
mc 0.11 0.01 0.03
mw 0.03 0.00 0.00
## ML
name Pd Pf G
jm 1.00 0.99 0.97
kc 0.46 0.48 0.43
cm 0.10 0.03 0.06
mc 0.09 0.01 0.02
mw 0.00 0.00 0.00
## JDT
name Pd Pf G
Zxin 0.47 0.42 0.46
Apac 0.50 0.10 0.17
Safe 0.35 0.05 0.09
## PDE
name Pd Pf G
Zxin 0.66 0.70 0.66
Apac 0.52 0.33 0.40
Safe 0.20 0.06 0.10
## EQ
name Pd Pf G
Zxin* 0.74 0.36 0.47
Apac 0.39 0.10 0.19
Safe 0.15 0.01 0.02
## LC
name Pd Pf G
Zxin 0.65 0.55 0.57
Apac 0.52 0.41 0.48
Safe 0.09 0.08 0.10
## ML
name Pd Pf G
Zxin 0.51 0.44 0.47
Safe 0.26 0.15 0.19
Apac 0.19 0.11 0.17
## ant
name Pd Pf G
Zxin 0.91 0.82 0.80
Apac 0.34 0.12 0.20
Safe 0.28 0.05 0.09
## xerces
name Pd Pf G
Zxin 0.53 0.43 0.48
Apac 0.38 0.47 0.44
Safe 0.07 0.07 0.08
## xalan
name Pd Pf G
Apac 0.37 0.33 0.34
Zxin 0.35 0.34 0.33
Safe 0.28 0.12 0.16
## camel
name Pd Pf G
Zxin 0.76 0.64 0.66
Apac 0.24 0.16 0.19
Safe 0.05 0.03 0.04
## log4j
name Pd Pf G
Apac 0.74 0.64 0.71
Safe 0.37 0.43 0.40
Zxin 0.21 0.19 0.24
## lucene
name Pd Pf G
Zxin 0.50 0.45 0.46
Apac 0.11 0.04 0.09
Safe 0.03 0.01 0.01
## jedit
name Pd Pf G
Apac 0.46 0.24 0.32
Zxin 0.36 0.15 0.21
Safe 0.09 0.03 0.04
## poi
name Pd Pf G
Zxin 0.59 0.60 0.57
Apac 0.36 0.43 0.43
Safe 0.28 0.12 0.17
## velocity
name Pd Pf G
Zxin 0.69 0.78 0.71
Apac 0.42 0.33 0.37
Safe 0.02 0.02 0.02
## ivy
name Pd Pf G
Zxin 0.77 0.73 0.74
Apac 0.53 0.24 0.44
Safe 0.19 0.13 0.16
[~]$ python -B ~/git/rahlk/Bellwether/src/TCA+/par_exec.py
Looked at Pd, Pf, G (where g = sqrt(Pd*(1-Pf)); the harmonic mean between Pd and Pf used for sorting)
No clear Bellwether on any of the metrics. We were able to find on a metric called Balance (which was the euclidean distance from ideal score (Pd, Pf) = (1, 0)). Our results
here.
Now, note that out round robin evaluation strategy of finding the best dataset significantly outperforms TCA+
The authors of TCA+ do not report results on this data set. Moreover, they conveniently breeze through the most important part, the TCA algorithm. See sections 3.1 and 3.2 (this was a beast to implement! But offered no major benefits).
Their primary contribution of TCA+ is a proposal on choosing the best normalization strategy.
(I'm sorry, this is a formatted raw dump. I'll create a summary table and paste it here.)
+----------+------+------+------+
| Name | Pd | Pf | G |
+==========+======+======+======+
| lucene | 0.60 | 0.60 | 0.49 |
+----------+------+------+------+
| poi | 0.59 | 0.60 | 0.48 |
+----------+------+------+------+
| velocity | 0.74 | 0.69 | 0.47 |
+----------+------+------+------+
| log4j | 0.69 | 0.69 | 0.46 |
+----------+------+------+------+
| xerces | 0.86 | 0.83 | 0.38 |
+----------+------+------+------+
| camel | 0.08 | 0.06 | 0.27 |
+----------+------+------+------+
| jedit | 0.06 | 0.09 | 0.23 |
+----------+------+------+------+
| ivy | 0.05 | 0.07 | 0.22 |
+----------+------+------+------+
| xalan | 0.03 | 0.03 | 0.17 |
+----------+------+------+------+
+----------+------+------+------+
| Name | Pd | Pf | G |
+==========+======+======+======+
| ivy | 0.67 | 0.25 | 0.71 |
+----------+------+------+------+
| jedit | 0.61 | 0.20 | 0.70 |
+----------+------+------+------+
| xalan | 0.72 | 0.32 | 0.70 |
+----------+------+------+------+
| lucene | 0.77 | 0.39 | 0.69 |
+----------+------+------+------+
| poi | 0.78 | 0.40 | 0.68 |
+----------+------+------+------+
| camel | 0.79 | 0.45 | 0.66 |
+----------+------+------+------+
| velocity | 0.65 | 0.34 | 0.66 |
+----------+------+------+------+
| log4j | 0.89 | 0.54 | 0.64 |
+----------+------+------+------+
| xerces | 0.83 | 0.53 | 0.63 |
+----------+------+------+------+
+----------+------+------+------+
| Name | Pd | Pf | G |
+==========+======+======+======+
| velocity | 0.76 | 0.78 | 0.41 |
+----------+------+------+------+
| poi | 0.28 | 0.47 | 0.39 |
+----------+------+------+------+
| log4j | 0.92 | 0.90 | 0.30 |
+----------+------+------+------+
| xerces | 0.09 | 0.07 | 0.29 |
+----------+------+------+------+
| ant | 0.05 | 0.08 | 0.22 |
+----------+------+------+------+
| camel | 0.05 | 0.03 | 0.22 |
+----------+------+------+------+
| jedit | 0.03 | 0.11 | 0.17 |
+----------+------+------+------+
| ivy | 0.03 | 0.03 | 0.17 |
+----------+------+------+------+
| xalan | 0.03 | 0.05 | 0.15 |
+----------+------+------+------+
+----------+------+------+------+
| Name | Pd | Pf | G |
+==========+======+======+======+
| camel | 0.58 | 0.27 | 0.65 |
+----------+------+------+------+
| log4j | 0.75 | 0.46 | 0.63 |
+----------+------+------+------+
| poi | 0.41 | 0.17 | 0.58 |
+----------+------+------+------+
| ivy | 0.39 | 0.17 | 0.57 |
+----------+------+------+------+
| xerces | 0.62 | 0.47 | 0.57 |
+----------+------+------+------+
| velocity | 0.42 | 0.26 | 0.56 |
+----------+------+------+------+
| xalan | 0.42 | 0.29 | 0.55 |
+----------+------+------+------+
| ant | 0.32 | 0.12 | 0.53 |
+----------+------+------+------+
| jedit | 0.22 | 0.06 | 0.45 |
+----------+------+------+------+
+----------+------+------+------+
| Name | Pd | Pf | G |
+==========+======+======+======+
| velocity | 0.48 | 0.46 | 0.51 |
+----------+------+------+------+
| poi | 0.53 | 0.55 | 0.49 |
+----------+------+------+------+
| lucene | 0.42 | 0.44 | 0.48 |
+----------+------+------+------+
| xerces | 0.81 | 0.80 | 0.41 |
+----------+------+------+------+
| xalan | 0.12 | 0.14 | 0.32 |
+----------+------+------+------+
| ivy | 0.10 | 0.12 | 0.30 |
+----------+------+------+------+
| log4j | 0.92 | 0.93 | 0.26 |
+----------+------+------+------+
| ant | 0.04 | 0.03 | 0.19 |
+----------+------+------+------+
| camel | 0.00 | 0.03 | 0.06 |
+----------+------+------+------+
+----------+------+------+------+
| Name | Pd | Pf | G |
+==========+======+======+======+
| ant | 0.68 | 0.26 | 0.71 |
+----------+------+------+------+
| xalan | 0.65 | 0.33 | 0.66 |
+----------+------+------+------+
| ivy | 0.51 | 0.17 | 0.65 |
+----------+------+------+------+
| lucene | 0.67 | 0.36 | 0.65 |
+----------+------+------+------+
| camel | 0.76 | 0.47 | 0.63 |
+----------+------+------+------+
| poi | 0.48 | 0.23 | 0.61 |
+----------+------+------+------+
| velocity | 0.62 | 0.44 | 0.59 |
+----------+------+------+------+
| log4j | 0.91 | 0.64 | 0.57 |
+----------+------+------+------+
| xerces | 0.80 | 0.61 | 0.56 |
+----------+------+------+------+
+----------+------+------+------+
| Name | Pd | Pf | G |
+==========+======+======+======+
| velocity | 0.52 | 0.56 | 0.48 |
+----------+------+------+------+
| lucene | 0.37 | 0.41 | 0.47 |
+----------+------+------+------+
| poi | 0.30 | 0.31 | 0.46 |
+----------+------+------+------+
| xalan | 0.24 | 0.27 | 0.42 |
+----------+------+------+------+
| log4j | 0.74 | 0.85 | 0.34 |
+----------+------+------+------+
| jedit | 0.03 | 0.02 | 0.17 |
+----------+------+------+------+
| ivy | 0.02 | 0.02 | 0.15 |
+----------+------+------+------+
| camel | 0.02 | 0.01 | 0.13 |
+----------+------+------+------+
| ant | 0.02 | 0.01 | 0.12 |
+----------+------+------+------+
+----------+------+------+------+
| Name | Pd | Pf | G |
+==========+======+======+======+
| log4j | 0.47 | 0.27 | 0.59 |
+----------+------+------+------+
| camel | 0.44 | 0.32 | 0.55 |
+----------+------+------+------+
| velocity | 0.41 | 0.26 | 0.55 |
+----------+------+------+------+
| xalan | 0.41 | 0.31 | 0.53 |
+----------+------+------+------+
| lucene | 0.38 | 0.33 | 0.50 |
+----------+------+------+------+
| poi | 0.31 | 0.21 | 0.50 |
+----------+------+------+------+
| ant | 0.17 | 0.09 | 0.39 |
+----------+------+------+------+
| jedit | 0.16 | 0.07 | 0.38 |
+----------+------+------+------+
| ivy | 0.14 | 0.07 | 0.36 |
+----------+------+------+------+
+----------+------+------+------+
| Name | Pd | Pf | G |
+==========+======+======+======+
| log4j | 0.47 | 0.37 | 0.55 |
+----------+------+------+------+
| lucene | 0.35 | 0.40 | 0.46 |
+----------+------+------+------+
| xerces | 0.70 | 0.70 | 0.46 |
+----------+------+------+------+
| poi | 0.33 | 0.35 | 0.46 |
+----------+------+------+------+
| velocity | 0.26 | 0.26 | 0.44 |
+----------+------+------+------+
| camel | 0.11 | 0.10 | 0.32 |
+----------+------+------+------+
| ivy | 0.10 | 0.13 | 0.30 |
+----------+------+------+------+
| jedit | 0.07 | 0.03 | 0.26 |
+----------+------+------+------+
| ant | 0.06 | 0.05 | 0.24 |
+----------+------+------+------+
+----------+------+------+------+
| Name | Pd | Pf | G |
+==========+======+======+======+
| lucene | 0.55 | 0.42 | 0.57 |
+----------+------+------+------+
| camel | 0.49 | 0.37 | 0.56 |
+----------+------+------+------+
| log4j | 0.55 | 0.43 | 0.56 |
+----------+------+------+------+
| xerces | 0.62 | 0.55 | 0.53 |
+----------+------+------+------+
| velocity | 0.34 | 0.26 | 0.50 |
+----------+------+------+------+
| ant | 0.27 | 0.14 | 0.49 |
+----------+------+------+------+
| poi | 0.29 | 0.19 | 0.48 |
+----------+------+------+------+
| ivy | 0.25 | 0.15 | 0.46 |
+----------+------+------+------+
| jedit | 0.23 | 0.13 | 0.45 |
+----------+------+------+------+
+----------+------+------+------+
| Name | Pd | Pf | G |
+==========+======+======+======+
| velocity | 0.74 | 0.62 | 0.53 |
+----------+------+------+------+
| log4j | 0.55 | 0.55 | 0.50 |
+----------+------+------+------+
| xalan | 0.59 | 0.61 | 0.48 |
+----------+------+------+------+
| lucene | 0.31 | 0.35 | 0.45 |
+----------+------+------+------+
| poi | 0.25 | 0.24 | 0.44 |
+----------+------+------+------+
| xerces | 0.86 | 0.85 | 0.36 |
+----------+------+------+------+
| jedit | 0.01 | 0.01 | 0.12 |
+----------+------+------+------+
| ant | 0.01 | 0.01 | 0.08 |
+----------+------+------+------+
| ivy | 0.01 | 0.02 | 0.08 |
+----------+------+------+------+
+----------+------+------+------+
| Name | Pd | Pf | G |
+==========+======+======+======+
| lucene | 0.51 | 0.33 | 0.59 |
+----------+------+------+------+
| xalan | 0.44 | 0.28 | 0.57 |
+----------+------+------+------+
| log4j | 0.67 | 0.53 | 0.56 |
+----------+------+------+------+
| velocity | 0.56 | 0.44 | 0.56 |
+----------+------+------+------+
| xerces | 0.60 | 0.54 | 0.53 |
+----------+------+------+------+
| ant | 0.28 | 0.14 | 0.49 |
+----------+------+------+------+
| poi | 0.29 | 0.20 | 0.49 |
+----------+------+------+------+
| ivy | 0.26 | 0.17 | 0.46 |
+----------+------+------+------+
| jedit | 0.18 | 0.07 | 0.41 |
+----------+------+------+------+
+--------+------+------+------+
| Name | Pd | Pf | G |
+========+======+======+======+
| lucene | 0.54 | 0.56 | 0.49 |
+--------+------+------+------+
| poi | 0.29 | 0.25 | 0.46 |
+--------+------+------+------+
| camel | 0.23 | 0.24 | 0.41 |
+--------+------+------+------+
| log4j | 0.77 | 0.80 | 0.39 |
+--------+------+------+------+
| xerces | 0.04 | 0.08 | 0.20 |
+--------+------+------+------+
| ivy | 0.03 | 0.02 | 0.18 |
+--------+------+------+------+
| ant | 0.02 | 0.07 | 0.13 |
+--------+------+------+------+
| jedit | 0.01 | 0.01 | 0.12 |
+--------+------+------+------+
| xalan | 0.01 | 0.03 | 0.10 |
+--------+------+------+------+
+--------+------+------+------+
| Name | Pd | Pf | G |
+========+======+======+======+
| lucene | 0.52 | 0.35 | 0.58 |
+--------+------+------+------+
| xalan | 0.47 | 0.29 | 0.58 |
+--------+------+------+------+
| log4j | 0.77 | 0.57 | 0.57 |
+--------+------+------+------+
| camel | 0.43 | 0.39 | 0.51 |
+--------+------+------+------+
| xerces | 0.47 | 0.44 | 0.51 |
+--------+------+------+------+
| ant | 0.29 | 0.15 | 0.49 |
+--------+------+------+------+
| poi | 0.31 | 0.24 | 0.49 |
+--------+------+------+------+
| ivy | 0.19 | 0.09 | 0.42 |
+--------+------+------+------+
| jedit | 0.11 | 0.07 | 0.31 |
+--------+------+------+------+
+----------+------+------+------+
| Name | Pd | Pf | G |
+==========+======+======+======+
| velocity | 0.56 | 0.55 | 0.50 |
+----------+------+------+------+
| lucene | 0.82 | 0.78 | 0.43 |
+----------+------+------+------+
| log4j | 0.89 | 0.86 | 0.35 |
+----------+------+------+------+
| xerces | 0.09 | 0.12 | 0.29 |
+----------+------+------+------+
| xalan | 0.07 | 0.14 | 0.25 |
+----------+------+------+------+
| ivy | 0.04 | 0.03 | 0.21 |
+----------+------+------+------+
| ant | 0.02 | 0.01 | 0.14 |
+----------+------+------+------+
| jedit | 0.02 | 0.02 | 0.12 |
+----------+------+------+------+
| camel | 0.01 | 0.00 | 0.08 |
+----------+------+------+------+
+----------+------+------+------+
| Name | Pd | Pf | G |
+==========+======+======+======+
| lucene | 0.73 | 0.40 | 0.66 |
+----------+------+------+------+
| camel | 0.75 | 0.51 | 0.61 |
+----------+------+------+------+
| ivy | 0.44 | 0.19 | 0.60 |
+----------+------+------+------+
| log4j | 0.85 | 0.59 | 0.59 |
+----------+------+------+------+
| xalan | 0.50 | 0.33 | 0.58 |
+----------+------+------+------+
| xerces | 0.42 | 0.34 | 0.53 |
+----------+------+------+------+
| ant | 0.27 | 0.13 | 0.48 |
+----------+------+------+------+
| velocity | 0.22 | 0.27 | 0.40 |
+----------+------+------+------+
| jedit | 0.14 | 0.05 | 0.36 |
+----------+------+------+------+
+----------+------+------+------+
| Name | Pd | Pf | G |
+==========+======+======+======+
| lucene | 0.46 | 0.32 | 0.56 |
+----------+------+------+------+
| xerces | 0.31 | 0.32 | 0.46 |
+----------+------+------+------+
| poi | 0.67 | 0.69 | 0.45 |
+----------+------+------+------+
| velocity | 0.86 | 0.89 | 0.30 |
+----------+------+------+------+
| ivy | 0.11 | 0.18 | 0.30 |
+----------+------+------+------+
| jedit | 0.07 | 0.04 | 0.25 |
+----------+------+------+------+
| ant | 0.03 | 0.03 | 0.18 |
+----------+------+------+------+
| xalan | 0.02 | 0.03 | 0.15 |
+----------+------+------+------+
| camel | 0.02 | 0.01 | 0.15 |
+----------+------+------+------+
+----------+------+------+------+
| Name | Pd | Pf | G |
+==========+======+======+======+
| camel | 0.52 | 0.21 | 0.64 |
+----------+------+------+------+
| poi | 0.38 | 0.10 | 0.59 |
+----------+------+------+------+
| velocity | 0.45 | 0.21 | 0.59 |
+----------+------+------+------+
| lucene | 0.36 | 0.08 | 0.57 |
+----------+------+------+------+
| xalan | 0.50 | 0.34 | 0.57 |
+----------+------+------+------+
| ant | 0.28 | 0.03 | 0.52 |
+----------+------+------+------+
| xerces | 0.66 | 0.59 | 0.52 |
+----------+------+------+------+
| ivy | 0.27 | 0.03 | 0.51 |
+----------+------+------+------+
| jedit | 0.19 | 0.02 | 0.43 |
+----------+------+------+------+
+----------+------+------+------+
| Name | Pd | Pf | G |
+==========+======+======+======+
| jedit | 0.39 | 0.22 | 0.55 |
+----------+------+------+------+
| poi | 0.30 | 0.19 | 0.50 |
+----------+------+------+------+
| lucene | 0.85 | 0.76 | 0.45 |
+----------+------+------+------+
| velocity | 0.72 | 0.73 | 0.44 |
+----------+------+------+------+
| log4j | 0.62 | 0.70 | 0.43 |
+----------+------+------+------+
| ant | 0.10 | 0.07 | 0.31 |
+----------+------+------+------+
| camel | 0.08 | 0.09 | 0.26 |
+----------+------+------+------+
| xerces | 0.06 | 0.07 | 0.23 |
+----------+------+------+------+
| xalan | 0.02 | 0.06 | 0.13 |
+----------+------+------+------+
+----------+------+------+------+
| Name | Pd | Pf | G |
+==========+======+======+======+
| lucene | 0.71 | 0.38 | 0.66 |
+----------+------+------+------+
| ant | 0.49 | 0.16 | 0.64 |
+----------+------+------+------+
| camel | 0.78 | 0.47 | 0.64 |
+----------+------+------+------+
| log4j | 0.84 | 0.51 | 0.64 |
+----------+------+------+------+
| poi | 0.53 | 0.24 | 0.64 |
+----------+------+------+------+
| xalan | 0.57 | 0.29 | 0.64 |
+----------+------+------+------+
| velocity | 0.69 | 0.46 | 0.61 |
+----------+------+------+------+
| jedit | 0.39 | 0.19 | 0.57 |
+----------+------+------+------+
| xerces | 0.75 | 0.78 | 0.40 |
+----------+------+------+------+
name,Pd,Pf,G
lucene, 0.88, 0.71, 0.39
xerces, 0.75, 0.57, 0.32
log4j, 0.83, 0.44, 0.29
xalan, 0.59, 0.34, 0.22
poi, 0.49, 0.18, 0.13
camel, 0.50, 0.10, 0.09
velocity, 0.38, 0.11, 0.08
ant, 0.09, 0.02, 0.02
ivy, 0.09, 0.01, 0.01
jedit, 0.03, 0.00, 0.00
name,Pd,Pf,G
velocity, 0.93, 0.99, 0.48
lucene, 0.60, 0.36, 0.23
log4j, 0.62, 0.29, 0.20
xerces, 0.30, 0.45, 0.18
poi, 0.32, 0.21, 0.12
ivy, 0.27, 0.09, 0.07
ant, 0.14, 0.09, 0.06
camel, 0.10, 0.10, 0.05
xalan, 0.11, 0.08, 0.05
jedit, 0.00, 0.00, 0.00
name,Pd,Pf,G
velocity, 0.85, 0.64, 0.37
log4j, 0.85, 0.54, 0.33
lucene, 0.88, 0.48, 0.31
poi, 0.87, 0.39, 0.27
xerces, 0.60, 0.43, 0.25
xalan, 0.80, 0.34, 0.24
camel, 0.39, 0.11, 0.09
ant, 0.09, 0.04, 0.03
jedit, 0.08, 0.01, 0.00
ivy, 0.05, 0.00, 0.00
name,Pd,Pf,G
lucene, 1.00, 0.94, 0.49
log4j, 0.95, 0.89, 0.46
velocity, 0.59, 0.53, 0.28
xerces, 0.55, 0.28, 0.18
xalan, 0.50, 0.29, 0.18
poi, 0.52, 0.16, 0.12
camel, 0.19, 0.07, 0.05
ivy, 0.17, 0.03, 0.03
ant, 0.03, 0.00, 0.00
jedit, 0.00, 0.00, 0.00
name,Pd,Pf,G
log4j, 0.71, 0.53, 0.30
lucene, 0.70, 0.44, 0.27
velocity, 0.69, 0.40, 0.25
xalan, 0.56, 0.32, 0.20
xerces, 0.30, 0.35, 0.16
ivy, 0.13, 0.07, 0.05
poi, 0.13, 0.05, 0.03
camel, 0.04, 0.01, 0.00
ant, 0.03, 0.00, 0.00
jedit, 0.00, 0.00, 0.00
name,Pd,Pf,G
JDT, 0.70, 0.23, 0.17
PDE, 0.40, 0.19, 0.13
EQ, 0.52, 0.15, 0.11
LC, 0.41, 0.16, 0.11
ML, 0.27, 0.09, 0.07
name,Pd,Pf,G
EQ, 0.86, 0.76, 0.41
PDE, 0.68, 0.41, 0.26
LC, 0.73, 0.29, 0.21
JDT, 0.68, 0.29, 0.21
ML, 0.55, 0.21, 0.15
name,Pd,Pf,G
EQ, 0.69, 0.60, 0.32
JDT, 0.58, 0.39, 0.23
PDE, 0.32, 0.35, 0.17
LC, 0.42, 0.22, 0.15
ML, 0.24, 0.11, 0.08
name,Pd,Pf,G
lucene, 0.96, 0.77, 0.43
log4j, 0.95, 0.75, 0.42
poi, 0.93, 0.65, 0.38
xalan, 0.78, 0.42, 0.27
xerces, 0.70, 0.44, 0.27
velocity, 0.54, 0.43, 0.24
camel, 0.27, 0.08, 0.06
ivy, 0.10, 0.02, 0.02
ant, 0.05, 0.02, 0.01
jedit, 0.02, 0.00, 0.00
name,Pd,Pf,G
log4j, 0.91, 0.82, 0.43
xalan, 0.91, 0.68, 0.39
lucene, 0.86, 0.56, 0.34
velocity, 0.59, 0.76, 0.33
xerces, 0.73, 0.53, 0.31
camel, 0.82, 0.44, 0.29
poi, 0.68, 0.38, 0.24
ant, 0.59, 0.26, 0.18
ivy, 0.55, 0.12, 0.10
jedit, 0.27, 0.15, 0.10
name,Pd,Pf,G
velocity, 0.86, 0.95, 0.45
lucene, 0.89, 0.72, 0.40
xerces, 0.78, 0.70, 0.37
poi, 0.81, 0.65, 0.36
xalan, 0.79, 0.57, 0.33
log4j, 0.72, 0.48, 0.29
camel, 0.33, 0.16, 0.11
ivy, 0.13, 0.10, 0.06
jedit, 0.14, 0.05, 0.03
ant, 0.10, 0.05, 0.03
name,Pd,Pf,G
jm, 0.80, 0.53, 0.32
kc, 0.70, 0.32, 0.22
cm, 0.31, 0.05, 0.04
mc, 0.02, 0.01, 0.01
mw, 0.00, 0.00, 0.00
name,Pd,Pf,G
jm, 0.86, 0.41, 0.28
kc, 0.32, 0.56, 0.20
cm, 0.45, 0.18, 0.13
mw, 0.14, 0.00, 0.00
mc, 0.14, 0.00, 0.00
name,Pd,Pf,G
jm, 0.64, 0.44, 0.26
kc, 0.41, 0.33, 0.18
cm, 0.08, 0.03, 0.02
mc, 0.03, 0.01, 0.01
mw, 0.00, 0.00, 0.00
name,Pd,Pf,G
jm, 0.54, 0.26, 0.17
kc, 0.05, 0.02, 0.01
cm, 0.01, 0.01, 0.00
mw, 0.00, 0.00, 0.00
mc, 0.00, 0.00, 0.00
name,Pd,Pf,G
jm, 0.45, 0.16, 0.12
kc, 0.07, 0.05, 0.03
cm, 0.05, 0.02, 0.01
mw, 0.00, 0.00, 0.00
mc, 0.00, 0.00, 0.00
name,Pd,Pf,G
jm, 0.67, 0.28, 0.20
cm, 0.13, 0.03, 0.02
kc, 0.10, 0.01, 0.00
mw, 0.00, 0.00, 0.00
mc, 0.05, 0.00, 0.00
name,Pd,Pf,G
kc, 0.80, 0.98, 0.44
jm, 0.69, 0.45, 0.27
cm, 0.16, 0.03, 0.03
mw, 0.02, 0.00, 0.00
mc, 0.02, 0.00, 0.00
name,Pd,Pf,G
jm, 0.26, 0.18, 0.11
kc, 0.26, 0.08, 0.06
cm, 0.04, 0.04, 0.02
mw, 0.00, 0.00, 0.00
mc, 0.00, 0.00, 0.00
name,Pd,Pf,G
Apache, 0.23, 0.05, 0.04
Safe, 0.23, 0.05, 0.04
Zxing, 0.07, 0.06, 0.03
name,Pd,Pf,G
Apache, 0.06, 0.07, 0.03
Safe, 0.05, 0.06, 0.03
Zxing, 0.00, 0.00, 0.00
name,Pd,Pf,G
Safe, 0.48, 0.24, 0.16
Apache, 0.35, 0.17, 0.11
Zxing, 0.04, 0.01, 0.01
name,Pd,Pf,G
Zxing, 0.22, 0.10, 0.07
Apache, 0.05, 0.09, 0.03
Safe, 0.19, 0.02, 0.02
name,Pd,Pf,G
Apache, 0.13, 0.19, 0.08
Safe, 0.23, 0.09, 0.06
Zxing, 0.07, 0.07, 0.03
name,Pd,Pf,G
JDT, 0.81, 0.49, 0.31
EQ, 0.54, 0.24, 0.17
LC, 0.26, 0.07, 0.05
PDE, 0.08, 0.05, 0.03
ML, 0.14, 0.03, 0.02
name,Pd,Pf,G
EQ, 0.37, 0.21, 0.14
PDE, 0.22, 0.21, 0.11
JDT, 0.19, 0.19, 0.10
LC, 0.22, 0.10, 0.07
ML, 0.03, 0.01, 0.00
name,Pd,Pf,G
PDE, 0.34, 0.26, 0.15
EQ, 0.32, 0.26, 0.14
JDT, 0.11, 0.07, 0.04
LC, 0.12, 0.05, 0.04
ML, 0.04, 0.02, 0.01
name,Pd,Pf,G
PDE, 0.27, 0.20, 0.12
LC, 0.28, 0.13, 0.09
EQ, 0.18, 0.15, 0.08
ML, 0.10, 0.06, 0.04
JDT, 0.09, 0.03, 0.02
name,Pd,Pf,G
JDT, 0.75, 0.66, 0.35
PDE, 0.54, 0.69, 0.30
EQ, 0.57, 0.48, 0.26
ML, 0.48, 0.33, 0.20
LC, 0.25, 0.03, 0.02
name,Pd,Pf,G
PDE, 0.52, 0.51, 0.26
EQ, 0.36, 0.16, 0.11
LC, 0.23, 0.10, 0.07
JDT, 0.09, 0.03, 0.02
ML, 0.05, 0.00, 0.00
name,Pd,Pf,G
PDE, 0.60, 0.44, 0.25
EQ, 0.59, 0.34, 0.22
LC, 0.20, 0.06, 0.05
JDT, 0.20, 0.06, 0.04
ML, 0.01, 0.01, 0.01
name,Pd,Pf,G
EQ, 0.91, 0.76, 0.42
PDE, 0.17, 0.18, 0.09
JDT, 0.09, 0.03, 0.02
LC, 0.03, 0.00, 0.00
ML, 0.01, 0.00, 0.00
name,Pd,Pf,G
LC, 0.28, 0.15, 0.10
JDT, 0.23, 0.15, 0.09
EQ, 0.19, 0.10, 0.06
PDE, 0.12, 0.07, 0.05
ML, 0.07, 0.03, 0.02
name,Pd,Pf,G
ML, 0.40, 0.25, 0.16
PDE, 0.28, 0.20, 0.12
JDT, 0.40, 0.16, 0.11
EQ, 0.39, 0.14, 0.11
LC, 0.22, 0.05, 0.04
name,Pd,Pf,G
kc, 0.10, 0.05, 0.03
jm, 0.02, 0.00, 0.00
cm, 0.01, 0.00, 0.00
mw, 0.00, 0.00, 0.00
mc, 0.00, 0.00, 0.00
name,Pd,Pf,G
jm, 0.39, 0.29, 0.17
kc, 0.30, 0.18, 0.11
cm, 0.05, 0.01, 0.00
mw, 0.01, 0.01, 0.00
mc, 0.01, 0.00, 0.00
name,Pd,Pf,G
jm, 0.55, 0.34, 0.21
kc, 0.07, 0.03, 0.02
cm, 0.02, 0.01, 0.01
mc, 0.01, 0.01, 0.00
mw, 0.00, 0.00, 0.00
name,Pd,Pf,G
kc, 0.56, 0.46, 0.25
jm, 0.40, 0.30, 0.17
cm, 0.00, 0.00, 0.00
mw, 0.00, 0.00, 0.00
mc, 0.00, 0.00, 0.00
name,Pd,Pf,G
jm, 0.78, 0.69, 0.37
kc, 0.35, 0.31, 0.17
cm, 0.31, 0.22, 0.13
mw, 0.00, 0.00, 0.00
mc, 0.00, 0.00, 0.00
name,Pd,Pf,G
jm, 0.77, 0.70, 0.37
kc, 0.04, 0.01, 0.01
cm, 0.02, 0.00, 0.00
mw, 0.00, 0.00, 0.00
mc, 0.00, 0.00, 0.00
name,Pd,Pf,G
jm, 0.38, 0.37, 0.19
kc, 0.05, 0.01, 0.01
mc, 0.04, 0.00, 0.00
cm, 0.00, 0.00, 0.00
mw, 0.00, 0.00, 0.00
name,Pd,Pf,G
kc, 0.76, 0.53, 0.31
jm, 0.51, 0.27, 0.18
mw, 0.00, 0.00, 0.00
cm, 0.00, 0.00, 0.00
mc, 0.00, 0.00, 0.00
name,Pd,Pf,G
kc, 0.34, 0.31, 0.16
jm, 0.10, 0.09, 0.05
cm, 0.05, 0.06, 0.03
mw, 0.01, 0.00, 0.00
mc, 0.00, 0.00, 0.00
name,Pd,Pf,G
jm, 0.22, 0.07, 0.05
cm, 0.02, 0.00, 0.00
kc, 0.00, 0.00, 0.00
mw, 0.00, 0.00, 0.00
mc, 0.01, 0.00, 0.00
name,Pd,Pf,G
Safe, 0.21, 0.21, 0.11
Zxing, 0.03, 0.11, 0.02
Apache, 0.01, 0.01, 0.00
name,Pd,Pf,G
Zxing, 0.41, 0.54, 0.23
Apache, 0.24, 0.16, 0.10
Safe, 0.17, 0.11, 0.07
name,Pd,Pf,G
Safe, 0.14, 0.15, 0.07
Zxing, 0.15, 0.12, 0.07
Apache, 0.05, 0.03, 0.02
name,Pd,Pf,G
Zxing, 0.28, 0.25, 0.13
Safe, 0.25, 0.22, 0.12
Apache, 0.29, 0.17, 0.11
name,Pd,Pf,G
Zxing, 0.14, 0.10, 0.06
Apache, 0.27, 0.03, 0.03
Safe, 0.01, 0.00, 0.00
name,Pd,Pf,G
Zxing, 0.32, 0.21, 0.13
Safe, 0.32, 0.20, 0.12
Apache, 0.30, 0.14, 0.10
name,Pd,Pf,G
Zxing, 0.19, 0.18, 0.09
Apache, 0.04, 0.04, 0.02
Safe, 0.06, 0.01, 0.01
name,Pd,Pf,G
Apache, 0.03, 0.02, 0.01
Safe, 0.02, 0.02, 0.01
Zxing, 0.00, 0.02, 0.00
name,Pd,Pf,G
Zxing, 0.25, 0.29, 0.13
Safe, 0.17, 0.13, 0.07
Apache, 0.06, 0.06, 0.03
name,Pd,Pf,G
Zxing, 0.05, 0.11, 0.04
Apache, 0.06, 0.02, 0.01
Safe, 0.06, 0.01, 0.01
name,Pd,Pf,G
PDE, 0.67, 0.53, 0.29
JDT, 0.56, 0.38, 0.23
EQ, 0.53, 0.34, 0.21
LC, 0.56, 0.32, 0.21
ML, 0.25, 0.30, 0.14
name,Pd,Pf,G
EQ, 0.16, 0.06, 0.05
PDE, 0.08, 0.06, 0.04
ML, 0.02, 0.00, 0.00
LC, 0.02, 0.00, 0.00
JDT, 0.02, 0.00, 0.00
name,Pd,Pf,G
EQ, 0.89, 0.78, 0.41
PDE, 0.85, 0.75, 0.40
JDT, 0.74, 0.50, 0.30
LC, 0.74, 0.43, 0.27
ML, 0.44, 0.21, 0.14
name,Pd,Pf,G
JDT, 0.52, 0.35, 0.21
EQ, 0.62, 0.30, 0.20
LC, 0.48, 0.22, 0.15
PDE, 0.33, 0.21, 0.13
ML, 0.24, 0.08, 0.06
name,Pd,Pf,G
JDT, 0.93, 0.84, 0.44
LC, 0.89, 0.68, 0.39
ML, 0.70, 0.32, 0.22
EQ, 0.63, 0.32, 0.21
PDE, 0.33, 0.24, 0.14
name,Pd,Pf,G
log4j, 0.78, 0.88, 0.41
lucene, 0.86, 0.61, 0.36
poi, 0.78, 0.64, 0.35
camel, 0.78, 0.59, 0.33
xalan, 0.72, 0.61, 0.33
xerces, 0.58, 0.54, 0.28
velocity, 0.39, 0.44, 0.21
ant, 0.22, 0.07, 0.06
ivy, 0.14, 0.04, 0.03
jedit, 0.08, 0.03, 0.02
name,Pd,Pf,G
velocity, 0.91, 0.85, 0.44
xerces, 0.50, 0.61, 0.28
log4j, 0.66, 0.45, 0.27
xalan, 0.42, 0.41, 0.21
lucene, 0.24, 0.10, 0.07
camel, 0.13, 0.03, 0.02
poi, 0.08, 0.02, 0.02
ivy, 0.03, 0.01, 0.01
ant, 0.01, 0.00, 0.00
jedit, 0.00, 0.00, 0.00
name,Pd,Pf,G
xalan, 1.00, 0.98, 0.49
poi, 0.96, 0.87, 0.46
log4j, 1.00, 0.84, 0.46
velocity, 1.00, 0.83, 0.45
lucene, 0.85, 0.68, 0.38
camel, 0.81, 0.59, 0.34
ant, 0.59, 0.64, 0.31
xerces, 0.78, 0.47, 0.29
ivy, 0.67, 0.35, 0.23
jedit, 0.63, 0.28, 0.19
name,Pd,Pf,G
xalan, 0.98, 0.91, 0.47
velocity, 0.93, 0.89, 0.45
poi, 0.98, 0.84, 0.45
lucene, 0.95, 0.78, 0.43
log4j, 0.90, 0.69, 0.39
xerces, 0.60, 0.71, 0.32
camel, 0.50, 0.29, 0.18
ivy, 0.45, 0.21, 0.14
ant, 0.24, 0.10, 0.07
jedit, 0.14, 0.08, 0.05
name,Pd,Pf,G
log4j, 1.00, 0.99, 0.50
lucene, 1.00, 0.96, 0.49
poi, 1.00, 0.92, 0.48
xalan, 0.96, 0.81, 0.44
xerces, 0.93, 0.79, 0.43
velocity, 0.63, 0.60, 0.31
camel, 0.56, 0.38, 0.23
jedit, 0.56, 0.33, 0.21
ivy, 0.56, 0.31, 0.20
ant, 0.48, 0.12, 0.09
name,Pd,Pf,G
Apache, 0.72, 0.57, 0.32
Zxing, 0.22, 0.37, 0.14
Safe, 0.17, 0.06, 0.05
name,Pd,Pf,G
Apache, 0.06, 0.03, 0.02
Zxing, 0.06, 0.02, 0.01
Safe, 0.02, 0.01, 0.01
name,Pd,Pf,G
Safe, 0.59, 0.36, 0.22
Zxing, 0.33, 0.32, 0.16
Apache, 0.22, 0.04, 0.04
name,Pd,Pf,G
Apache, 0.52, 0.36, 0.21
Safe, 0.17, 0.09, 0.06
Zxing, 0.00, 0.00, 0.00
name,Pd,Pf,G
Safe, 0.67, 0.30, 0.21
Apache, 0.41, 0.18, 0.13
Zxing, 0.15, 0.12, 0.07
/Users/rkrsn/git/rahlk/Bellwether/src/SEER/pickles/pretty_print.py
## JDT
name Pd Pf G
log4 0.94 0.77 0.85
xala 0.81 0.86 0.83
xerc 0.82 0.69 0.75
luce 0.82 0.50 0.62
velo* 0.72 0.45 0.56
poi 0.56 0.10 0.17
came 0.47 0.09 0.16
ivy 0.08 0.01 0.02
ant 0.12 0.00 0.01
jedi 0.01 0.00 0.00
## PDE
name Pd Pf G
luce 0.93 0.89 0.91
velo 0.82 0.91 0.86
log4 0.87 0.64 0.74
poi 0.78 0.63 0.70
xala 0.80 0.58 0.67
xerc 0.38 0.16 0.22
ivy 0.06 0.03 0.04
came 0.03 0.01 0.01
ant 0.00 0.00 0.00
jedi 0.00 0.00 0.00
## EQ
name Pd Pf G
velo 0.88 0.95 0.92
xerc 0.56 0.67 0.61
xala* 0.74 0.45 0.56
log4* 0.84 0.32 0.46
luce* 0.69 0.28 0.40
poi* 0.63 0.22 0.32
came 0.42 0.15 0.22
ivy 0.16 0.06 0.09
jedi 0.12 0.01 0.02
ant 0.07 0.01 0.01
## LC
name Pd Pf G
luce 1.00 0.87 0.93
log4 0.92 0.84 0.88
velo 0.88 0.77 0.82
xerc 0.84 0.79 0.81
xala 0.89 0.50 0.64
came 0.53 0.49 0.51
poi 0.42 0.09 0.15
ivy 0.03 0.03 0.03
jedi 0.03 0.01 0.01
ant 0.03 0.00 0.00
## ML
name Pd Pf G
xala 0.77 0.52 0.62
log4 0.63 0.55 0.59
velo 0.47 0.31 0.38
luce 0.45 0.20 0.28
xerc 0.18 0.34 0.23
poi 0.18 0.10 0.13
ivy 0.11 0.04 0.05
jedi 0.13 0.02 0.04
ant 0.07 0.02 0.03
came 0.04 0.02 0.03
## Apache
name Pd Pf G
LC 0.41 0.19 0.26
EQ 0.33 0.20 0.25
JDT 0.44 0.17 0.24
ML 0.51 0.15 0.23
PDE 0.06 0.04 0.05
## Safe
name Pd Pf G
EQ 0.86 0.68 0.76
PDE* 0.64 0.44 0.52
LC * 0.73 0.35 0.48
JDT* 0.77 0.32 0.46
ML 0.55 0.12 0.19
## Zxing
name Pd Pf G
JDT* 0.62 0.39 0.48
EQ * 0.60 0.39 0.47
PDE 0.50 0.38 0.43
ML 0.31 0.23 0.27
LC 0.36 0.20 0.26
## Apache
name Pd Pf G
luce 0.91 0.65 0.75
log4 0.83 0.55 0.66
xala 0.68 0.53 0.60
xerc* 0.78 0.46 0.58
poi* 0.80 0.38 0.51
came* 0.81 0.34 0.48
velo 0.36 0.51 0.42
ivy 0.57 0.24 0.34
jedi 0.13 0.02 0.04
ant 0.04 0.01 0.02
## Safe
name Pd Pf G
xala 0.91 0.82 0.86
luce 0.86 0.62 0.72
xerc 0.86 0.56 0.68
poi 0.82 0.56 0.66
velo 0.59 0.68 0.63
came* 0.77 0.41 0.54
log4* 0.82 0.38 0.52
jedi* 0.64 0.35 0.45
ivy* 0.68 0.24 0.35
ant* 0.64 0.24 0.34
## Zxing
name Pd Pf G
velo 0.99 0.90 0.94
log4 0.81 0.79 0.80
xala 0.86 0.72 0.78
poi 0.81 0.66 0.73
luce 0.77 0.51 0.62
xerc 0.64 0.59 0.61
came* 0.67 0.41 0.51
ivy 0.34 0.16 0.22
ant 0.11 0.04 0.06
jedi 0.02 0.01 0.01
## Apache
name Pd Pf G
jm * 0.67 0.39 0.49
mc 0.14 0.24 0.18
cm 0.42 0.10 0.17
kc 0.07 0.05 0.06
mw 0.00 0.00 0.00
## Safe
name Pd Pf G
jm * 0.86 0.35 0.50
mc 0.32 0.15 0.20
kc 0.00 0.00 0.00
mw 0.14 0.00 0.00
cm 0.05 0.00 0.00
## Zxing
name Pd Pf G
jm 0.58 0.38 0.46
cm 0.08 0.07 0.08
mc 0.02 0.08 0.03
kc 0.02 0.06 0.03
mw 0.01 0.00 0.01
## JDT
name Pd Pf G
jm 0.75 0.66 0.70
kc 0.20 0.04 0.07
mc 0.21 0.02 0.04
cm 0.02 0.01 0.01
mw 0.00 0.00 0.00
## PDE
name Pd Pf G
jm 0.43 0.32 0.37
kc 0.06 0.03 0.04
mc 0.10 0.01 0.02
cm 0.00 0.01 0.01
mw 0.00 0.00 0.00
## EQ
name Pd Pf G
jm * 0.69 0.37 0.48
kc 0.48 0.15 0.23
cm 0.04 0.01 0.01
mc 0.01 0.01 0.01
mw 0.02 0.00 0.00
## LC
name Pd Pf G
jm * 0.73 0.39 0.51
mc 0.03 0.03 0.03
kc 0.05 0.01 0.01
mw 0.02 0.00 0.00
cm 0.00 0.00 0.00
## ML
name Pd Pf G
jm 0.20 0.10 0.13
cm 0.02 0.01 0.01
kc 0.01 0.00 0.00
mc 0.02 0.00 0.00
mw 0.00 0.00 0.00
## JDT
name Pd Pf G
Safe 0.36 0.20 0.26
Apac 0.07 0.02 0.03
Zxin 0.10 0.01 0.01
## PDE
name Pd Pf G
Safe 0.47 0.30 0.37
Apac 0.29 0.14 0.19
Zxin 0.03 0.01 0.01
## EQ
name Pd Pf G
Apac 0.16 0.18 0.17
Safe 0.21 0.04 0.06
Zxin 0.07 0.04 0.05
## LC
name Pd Pf G
Apac 0.34 0.16 0.22
Zxin 0.08 0.02 0.04
Safe 0.08 0.02 0.03
## ML
name Pd Pf G
Safe 0.25 0.18 0.21
Zxin 0.27 0.17 0.21
Apac 0.06 0.04 0.05
## ant
name Pd Pf G
LC * 0.67 0.22 0.33
EQ 0.54 0.22 0.32
PDE 0.21 0.37 0.27
JDT 0.32 0.08 0.13
ML 0.09 0.01 0.02
## xerces
name Pd Pf G
PDE 0.67 0.74 0.71
LC 0.22 0.10 0.14
JDT 0.13 0.10 0.11
EQ 0.12 0.07 0.09
ML 0.04 0.03 0.04
## xalan
name Pd Pf G
PDE 0.70 0.67 0.69
JDT 0.73 0.57 0.64
EQ 0.55 0.49 0.52
LC 0.25 0.09 0.13
ML 0.04 0.01 0.02
## camel
name Pd Pf G
EQ 0.57 0.42 0.48
PDE 0.27 0.20 0.23
ML 0.10 0.06 0.07
LC 0.13 0.04 0.06
JDT 0.05 0.01 0.01
## log4j
name Pd Pf G
EQ * 0.62 0.48 0.54
LC 0.50 0.22 0.31
PDE 0.13 0.15 0.14
JDT 0.25 0.10 0.14
ML 0.02 0.00 0.00
## lucene
name Pd Pf G
JDT 0.40 0.14 0.20
PDE 0.13 0.20 0.16
LC 0.18 0.05 0.08
EQ 0.17 0.05 0.08
ML 0.08 0.03 0.05
## jedit
name Pd Pf G
EQ 0.60 0.34 0.43
JDT 0.53 0.17 0.25
PDE 0.24 0.13 0.17
ML 0.17 0.11 0.13
LC 0.33 0.08 0.13
## poi
name Pd Pf G
PDE 0.21 0.25 0.23
EQ 0.14 0.15 0.15
JDT 0.06 0.03 0.04
ML 0.06 0.02 0.03
LC 0.02 0.01 0.01
## velocity
name Pd Pf G
EQ 0.49 0.41 0.45
PDE 0.14 0.16 0.15
ML 0.11 0.09 0.10
JDT 0.14 0.07 0.09
LC 0.07 0.05 0.06
## ivy
name Pd Pf G
EQ 0.93 0.73 0.82
PDE 0.29 0.43 0.34
LC 0.42 0.15 0.23
ML 0.11 0.04 0.06
JDT 0.18 0.03 0.05
## ant
name Pd Pf G
jm * 0.66 0.30 0.41
kc 0.44 0.18 0.26
mc 0.09 0.02 0.04
mw 0.02 0.00 0.00
cm 0.00 0.00 0.00
## xerces
name Pd Pf G
jm 0.62 0.52 0.57
kc 0.18 0.15 0.16
mc 0.08 0.04 0.05
cm 0.02 0.00 0.01
mw 0.00 0.00 0.00
## xalan
name Pd Pf G
jm 0.33 0.24 0.28
mc 0.02 0.00 0.00
mw 0.01 0.00 0.00
kc 0.00 0.00 0.00
cm 0.00 0.00 0.00
## camel
name Pd Pf G
kc 0.31 0.23 0.27
jm 0.30 0.19 0.23
cm 0.02 0.01 0.01
mc 0.01 0.00 0.01
mw 0.00 0.00 0.00
## log4j
name Pd Pf G
jm 0.50 0.33 0.40
mc 0.11 0.04 0.06
cm 0.09 0.03 0.05
mw 0.05 0.04 0.04
kc 0.07 0.01 0.02
## lucene
name Pd Pf G
mc 0.19 0.15 0.17
jm 0.21 0.07 0.11
kc 0.02 0.01 0.01
cm 0.01 0.00 0.00
mw 0.00 0.00 0.00
## jedit
name Pd Pf G
jm 0.47 0.14 0.22
kc 0.02 0.01 0.01
cm 0.04 0.01 0.01
mc 0.02 0.00 0.00
mw 0.00 0.00 0.00
## poi
name Pd Pf G
jm 0.37 0.20 0.26
mc 0.12 0.06 0.08
kc 0.08 0.02 0.03
mw 0.01 0.00 0.00
cm 0.01 0.00 0.00
## velocity
name Pd Pf G
jm 0.49 0.35 0.41
mc 0.45 0.36 0.40
kc 0.32 0.19 0.24
mw 0.01 0.00 0.00
cm 0.00 0.00 0.00
## ivy
name Pd Pf G
kc 0.42 0.34 0.38
mc 0.42 0.28 0.33
jm 0.39 0.17 0.24
cm 0.03 0.00 0.00
mw 0.01 0.00 0.00
## ant
name Pd Pf G
Safe 0.49 0.26 0.34
Zxin 0.24 0.37 0.29
Apac 0.01 0.01 0.01
## xerces
name Pd Pf G
Safe 0.38 0.38 0.38
Zxin 0.33 0.29 0.31
Apac 0.17 0.11 0.14
## xalan
name Pd Pf G
Apac 0.22 0.20 0.21
Zxin 0.21 0.20 0.21
Safe 0.13 0.06 0.08
## camel
name Pd Pf G
Apac 0.30 0.29 0.29
Zxin 0.06 0.07 0.07
Safe 0.02 0.01 0.01
## log4j
name Pd Pf G
Zxin 0.08 0.07 0.08
Apac 0.04 0.10 0.06
Safe 0.02 0.01 0.01
## lucene
name Pd Pf G
Zxin 0.22 0.09 0.13
Apac 0.03 0.01 0.02
Safe 0.04 0.00 0.00
## jedit
name Pd Pf G
Apac 0.36 0.15 0.21
Safe 0.02 0.01 0.01
Zxin 0.00 0.00 0.00
## poi
name Pd Pf G
Zxin 0.12 0.12 0.12
Apac 0.11 0.04 0.06
Safe 0.04 0.07 0.05
## velocity
name Pd Pf G
Safe 0.23 0.17 0.19
Apac 0.13 0.08 0.10
Zxin 0.01 0.01 0.01
## ivy
name Pd Pf G
Apac 0.13 0.09 0.10
Safe 0.13 0.07 0.10
Zxin 0.09 0.06 0.08
## kc
name Pd Pf G
EQ 0.78 0.66 0.72
JDT 0.56 0.39 0.46
LC 0.56 0.24 0.33
PDE 0.42 0.18 0.25
ML 0.06 0.01 0.02
## jm
name Pd Pf G
PDE 0.04 0.02 0.03
EQ 0.03 0.00 0.01
ML 0.01 0.00 0.01
LC 0.01 0.00 0.00
JDT 0.01 0.00 0.00
## mw
name Pd Pf G
EQ 0.96 0.89 0.92
JDT 0.89 0.68 0.77
PDE* 0.67 0.44 0.53
LC * 0.70 0.37 0.49
ML 0.59 0.24 0.35
## cm
name Pd Pf G
PDE 0.79 0.65 0.71
JDT 0.36 0.18 0.24
LC 0.31 0.13 0.19
EQ 0.33 0.10 0.15
ML 0.21 0.07 0.10
## mc
name Pd Pf G
PDE 0.75 0.75 0.75
JDT 0.78 0.60 0.68
EQ 0.58 0.41 0.48
LC * 0.61 0.29 0.40
ML 0.53 0.29 0.38
## kc
name Pd Pf G
velo 0.97 0.97 0.97
came 1.00 0.84 0.91
log4 0.97 0.84 0.90
xala 0.89 0.73 0.80
luce 0.86 0.69 0.77
xerc 0.78 0.71 0.74
poi 0.67 0.58 0.62
jedi 0.31 0.14 0.20
ivy 0.33 0.11 0.17
ant 0.19 0.14 0.16
## jm
name Pd Pf G
velo 0.84 0.68 0.75
came* 0.71 0.48 0.57
xerc 0.43 0.62 0.51
log4 0.36 0.32 0.34
poi 0.47 0.25 0.32
luce 0.37 0.16 0.22
xala 0.23 0.17 0.19
ivy 0.00 0.00 0.00
ant 0.00 0.00 0.00
jedi 0.00 0.00 0.00
## mw
name Pd Pf G
log4 1.00 0.97 0.98
velo 0.96 0.94 0.95
poi 1.00 0.90 0.94
luce 0.89 0.85 0.87
came 0.89 0.81 0.85
xerc 0.89 0.70 0.79
xala 0.85 0.67 0.75
ant* 0.74 0.34 0.46
jedi 0.41 0.41 0.41
ivy 0.44 0.19 0.27
## cm
name Pd Pf G
log4 0.95 0.87 0.91
xala 0.93 0.81 0.86
velo 0.93 0.73 0.82
poi 0.86 0.63 0.73
luce* 0.81 0.42 0.55
xerc* 0.64 0.42 0.51
came 0.45 0.16 0.23
jedi 0.14 0.05 0.08
ant 0.07 0.03 0.05
ivy 0.10 0.03 0.04
## mc
name Pd Pf G
log4 0.97 0.91 0.94
poi 0.97 0.90 0.93
luce 0.89 0.74 0.81
velo 0.78 0.83 0.80
xala 0.89 0.66 0.76
xerc 0.78 0.60 0.67
came 0.56 0.37 0.44
ivy 0.39 0.15 0.22
ant 0.11 0.04 0.06
jedi 0.17 0.03 0.04
## kc
name Pd Pf G
Apac 0.47 0.31 0.38
Zxin 0.36 0.37 0.37
Safe 0.42 0.22 0.29
## jm
name Pd Pf G
Apac 0.02 0.00 0.00
Zxin 0.00 0.00 0.00
Safe 0.01 0.00 0.00
## mw
name Pd Pf G
Safe 0.52 0.30 0.38
Apac 0.04 0.09 0.05
Zxin 0.00 0.00 0.00
## cm
name Pd Pf G
Zxin 0.29 0.19 0.23
Safe 0.07 0.06 0.06
Apac 0.10 0.04 0.05
## mc
name Pd Pf G
Safe 0.36 0.16 0.22
Zxin 0.03 0.11 0.04
Apac 0.08 0.01 0.01
I use pycharm search for "holdout" in whole project, but it seems no holdout dataset that claimed in your paper in detect prediction module.
In old/ folder, the syntax seems to be Python2 but in other folder, code seems like Python3.
The question is, when run the run_all.py script, both python 2 and python 3 will be called, so there are so much syntax errors reported.
I'm a student from NJUS who want to reproduce your code. Thanks!
Email: [email protected]
____ ___ _ _ __ __ ___ ______ __ __ ___ ____
| \ / _]| | | | | |__| | / _] || | | / _]| \
| o ) / [_ | | | | | | | | / [_| || | | / [_ | D )
| || _]| |___ | |___ | | | || _]_| |_|| _ || _]| /
| O || [_ | || || ` ' || [_ | | | | || [_ | \
| || || || | \ / | | | | | | || || . \
|_____||_____||_____||_____| \_/\_/ |_____| |__| |__|__||_____||__|\_|
^
(\\
.--,\\__
`-. a`-.__
| ')
/ \ _.-'-,`;
/ | { /
..-"``~"-' ; )
;' `
;'
;'
, /;'|
,;';\ |\ |
\ || |
| )| )
| || |
jgs | \| \
`##`##
### ant
+----------+------+------+------+------+
| Dataset | G2 | ED | Pd | Pf |
+==========+======+======+======+======+
| ivy | 0.71 | 0.33 | 0.65 | 0.22 |
+----------+------+------+------+------+
| lucene | 0.71 | 0.29 | 0.81 | 0.38 |
+----------+------+------+------+------+
| poi | 0.70 | 0.32 | 0.69 | 0.29 |
+----------+------+------+------+------+
| jedit | 0.69 | 0.37 | 0.57 | 0.16 |
+----------+------+------+------+------+
| camel | 0.67 | 0.32 | 0.81 | 0.44 |
+----------+------+------+------+------+
| velocity | 0.67 | 0.33 | 0.83 | 0.46 |
+----------+------+------+------+------+
| xalan | 0.66 | 0.33 | 0.82 | 0.47 |
+----------+------+------+------+------+
| log4j | 0.65 | 0.34 | 0.87 | 0.51 |
+----------+------+------+------+------+
| xerces | 0.62 | 0.37 | 0.86 | 0.55 |
+----------+------+------+------+------+
### camel
+----------+------+------+------+------+
| Dataset | G2 | ED | Pd | Pf |
+==========+======+======+======+======+
| lucene | 0.58 | 0.44 | 0.60 | 0.44 |
+----------+------+------+------+------+
| xerces | 0.57 | 0.44 | 0.62 | 0.48 |
+----------+------+------+------+------+
| log4j | 0.56 | 0.44 | 0.64 | 0.50 |
+----------+------+------+------+------+
| velocity | 0.55 | 0.48 | 0.51 | 0.40 |
+----------+------+------+------+------+
| xalan | 0.55 | 0.51 | 0.44 | 0.31 |
+----------+------+------+------+------+
| poi | 0.53 | 0.56 | 0.35 | 0.21 |
+----------+------+------+------+------+
| ant | 0.48 | 0.62 | 0.27 | 0.17 |
+----------+------+------+------+------+
| ivy | 0.48 | 0.61 | 0.28 | 0.15 |
+----------+------+------+------+------+
| jedit | 0.36 | 0.73 | 0.13 | 0.05 |
+----------+------+------+------+------+
### ivy
+----------+------+------+------+------+
| Dataset | G2 | ED | Pd | Pf |
+==========+======+======+======+======+
| lucene | 0.66 | 0.35 | 0.69 | 0.38 |
+----------+------+------+------+------+
| ant | 0.64 | 0.43 | 0.50 | 0.16 |
+----------+------+------+------+------+
| xalan | 0.64 | 0.40 | 0.59 | 0.31 |
+----------+------+------+------+------+
| velocity | 0.63 | 0.37 | 0.70 | 0.43 |
+----------+------+------+------+------+
| xerces | 0.63 | 0.37 | 0.71 | 0.44 |
+----------+------+------+------+------+
| poi | 0.60 | 0.48 | 0.45 | 0.19 |
+----------+------+------+------+------+
| log4j | 0.59 | 0.39 | 0.80 | 0.56 |
+----------+------+------+------+------+
| jedit | 0.56 | 0.54 | 0.36 | 0.12 |
+----------+------+------+------+------+
| camel | 0.55 | 0.44 | 0.67 | 0.55 |
+----------+------+------+------+------+
### jedit
+----------+------+------+------+------+
| Dataset | G2 | ED | Pd | Pf |
+==========+======+======+======+======+
| ant | 0.71 | 0.32 | 0.65 | 0.22 |
+----------+------+------+------+------+
| poi | 0.70 | 0.34 | 0.64 | 0.24 |
+----------+------+------+------+------+
| xalan | 0.66 | 0.33 | 0.79 | 0.45 |
+----------+------+------+------+------+
| log4j | 0.63 | 0.36 | 0.83 | 0.52 |
+----------+------+------+------+------+
| lucene | 0.62 | 0.37 | 0.81 | 0.52 |
+----------+------+------+------+------+
| ivy | 0.59 | 0.50 | 0.42 | 0.16 |
+----------+------+------+------+------+
| camel | 0.58 | 0.42 | 0.65 | 0.47 |
+----------+------+------+------+------+
| xerces | 0.57 | 0.41 | 0.83 | 0.61 |
+----------+------+------+------+------+
| velocity | 0.51 | 0.51 | 0.52 | 0.49 |
+----------+------+------+------+------+
### log4j
+----------+------+------+------+------+
| Dataset | G2 | ED | Pd | Pf |
+==========+======+======+======+======+
| xerces | 0.62 | 0.42 | 0.58 | 0.34 |
+----------+------+------+------+------+
| velocity | 0.61 | 0.41 | 0.63 | 0.42 |
+----------+------+------+------+------+
| lucene | 0.57 | 0.55 | 0.35 | 0.08 |
+----------+------+------+------+------+
| ant | 0.55 | 0.56 | 0.33 | 0.11 |
+----------+------+------+------+------+
| camel | 0.55 | 0.46 | 0.60 | 0.49 |
+----------+------+------+------+------+
| ivy | 0.55 | 0.58 | 0.31 | 0.04 |
+----------+------+------+------+------+
| poi | 0.55 | 0.57 | 0.32 | 0.04 |
+----------+------+------+------+------+
| xalan | 0.53 | 0.50 | 0.52 | 0.45 |
+----------+------+------+------+------+
| jedit | 0.44 | 0.67 | 0.20 | 0.02 |
+----------+------+------+------+------+
### lucene
+----------+------+------+------+------+
| Dataset | G2 | ED | Pd | Pf |
+==========+======+======+======+======+
| log4j | 0.63 | 0.37 | 0.71 | 0.43 |
+----------+------+------+------+------+
| xerces | 0.62 | 0.41 | 0.59 | 0.35 |
+----------+------+------+------+------+
| velocity | 0.61 | 0.39 | 0.68 | 0.46 |
+----------+------+------+------+------+
| poi | 0.60 | 0.47 | 0.45 | 0.19 |
+----------+------+------+------+------+
| xalan | 0.57 | 0.50 | 0.42 | 0.23 |
+----------+------+------+------+------+
| camel | 0.55 | 0.47 | 0.57 | 0.47 |
+----------+------+------+------+------+
| ivy | 0.55 | 0.55 | 0.35 | 0.14 |
+----------+------+------+------+------+
| ant | 0.48 | 0.63 | 0.25 | 0.08 |
+----------+------+------+------+------+
| jedit | 0.46 | 0.64 | 0.23 | 0.07 |
+----------+------+------+------+------+
### poi
+----------+------+------+------+------+
| Dataset | G2 | ED | Pd | Pf |
+==========+======+======+======+======+
| lucene | 0.63 | 0.37 | 0.76 | 0.48 |
+----------+------+------+------+------+
| log4j | 0.61 | 0.38 | 0.81 | 0.55 |
+----------+------+------+------+------+
| xerces | 0.58 | 0.40 | 0.86 | 0.60 |
+----------+------+------+------+------+
| xalan | 0.55 | 0.51 | 0.43 | 0.29 |
+----------+------+------+------+------+
| ivy | 0.52 | 0.57 | 0.34 | 0.20 |
+----------+------+------+------+------+
| velocity | 0.51 | 0.54 | 0.43 | 0.39 |
+----------+------+------+------+------+
| ant | 0.48 | 0.62 | 0.26 | 0.11 |
+----------+------+------+------+------+
| camel | 0.48 | 0.56 | 0.44 | 0.46 |
+----------+------+------+------+------+
| jedit | 0.40 | 0.69 | 0.17 | 0.07 |
+----------+------+------+------+------+
### velocity
+---------+------+------+------+------+
| Dataset | G2 | ED | Pd | Pf |
+=========+======+======+======+======+
| ant | 0.57 | 0.51 | 0.42 | 0.22 |
+---------+------+------+------+------+
| camel | 0.57 | 0.44 | 0.60 | 0.46 |
+---------+------+------+------+------+
| lucene | 0.57 | 0.46 | 0.54 | 0.40 |
+---------+------+------+------+------+
| log4j | 0.55 | 0.51 | 0.44 | 0.31 |
+---------+------+------+------+------+
| xalan | 0.54 | 0.51 | 0.45 | 0.34 |
+---------+------+------+------+------+
| xerces | 0.51 | 0.48 | 0.64 | 0.59 |
+---------+------+------+------+------+
| ivy | 0.46 | 0.63 | 0.25 | 0.15 |
+---------+------+------+------+------+
| poi | 0.36 | 0.71 | 0.16 | 0.19 |
+---------+------+------+------+------+
| jedit | 0.32 | 0.75 | 0.11 | 0.06 |
+---------+------+------+------+------+
### xalan
+----------+------+------+------+------+
| Dataset | G2 | ED | Pd | Pf |
+==========+======+======+======+======+
| log4j | 0.56 | 0.48 | 0.51 | 0.39 |
+----------+------+------+------+------+
| camel | 0.52 | 0.54 | 0.41 | 0.32 |
+----------+------+------+------+------+
| lucene | 0.52 | 0.49 | 0.58 | 0.53 |
+----------+------+------+------+------+
| velocity | 0.52 | 0.54 | 0.40 | 0.33 |
+----------+------+------+------+------+
| ant | 0.50 | 0.59 | 0.31 | 0.18 |
+----------+------+------+------+------+
| poi | 0.48 | 0.61 | 0.28 | 0.17 |
+----------+------+------+------+------+
| xerces | 0.48 | 0.51 | 0.63 | 0.63 |
+----------+------+------+------+------+
| ivy | 0.47 | 0.62 | 0.28 | 0.19 |
+----------+------+------+------+------+
| jedit | 0.44 | 0.65 | 0.23 | 0.13 |
+----------+------+------+------+------+
### xerces
+----------+------+------+------+------+
| Dataset | G2 | ED | Pd | Pf |
+==========+======+======+======+======+
| log4j | 0.55 | 0.52 | 0.41 | 0.27 |
+----------+------+------+------+------+
| lucene | 0.55 | 0.52 | 0.41 | 0.26 |
+----------+------+------+------+------+
| camel | 0.52 | 0.53 | 0.42 | 0.36 |
+----------+------+------+------+------+
| xalan | 0.52 | 0.56 | 0.35 | 0.23 |
+----------+------+------+------+------+
| poi | 0.48 | 0.61 | 0.28 | 0.19 |
+----------+------+------+------+------+
| velocity | 0.48 | 0.59 | 0.34 | 0.30 |
+----------+------+------+------+------+
| ivy | 0.43 | 0.66 | 0.22 | 0.14 |
+----------+------+------+------+------+
| ant | 0.36 | 0.73 | 0.13 | 0.04 |
+----------+------+------+------+------+
| jedit | 0.36 | 0.72 | 0.13 | 0.05 |
+----------+------+------+------+------+
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.