Question about performance

[thunlp]

Question about performance

[wzq016]

The experiment was done a year ago and I didn't record the split that time, so I don't know the specific split I used then.

But it is not hard to get a reasonable split, all you need to do is to be patient and split the data until DMCNN's performance is relative stable.

I just uploaded one feasible split and my training logs, you should get similar results using such split.

[rz-zhang]

The experiment was done a year ago and I didn't record the split that time, so I don't know the specific split I used then.

But it is not hard to get a reasonable split, all you need to do is to be patient and split the data until DMCNN's performance is relative stable.

I just uploaded one feasible split and my training logs, you should get similar results using such split.

Thank you for the information. The proposed hierarchy module in this paper is very interesting. I also implemented your released code.

The following performance is averaged by 5 times:
DMCNN: 0.5155
HMEAE(CNN): 0.5019

I also wonder the data split and I am a little confused about "split the data until DMCNN's performance is relatively stable" in your reply.
Could you kindly provide more specific information to achieve the results as reported?

Thanks.

[Bakser]

We have provided a feasible split in https://github.com/thunlp/HMEAE/blob/master/logs/split.json. You can try it.
Another issue is that the original ACE 2005 files have a few obvious errors such as wrong span boundaries, different offsets for the same words, which cannot be directly handled by the script. As we cannot directly provide the cleaned files due to the license limit, we recommend to manually check the processed files while the number of errors is small.