lavis-nlp/contest

This repository provides evaluation code and link to our ConTest dataset.

contest's Issues

Issues while accessing via HuggingFace Datasets

The following error is thrown while loading from HuggingFace Datasets,

from datasets import load_dataset
raw_dataset = load_dataset('jdepoix/junit_test_completion', 'raw')

FileNotFoundError: Couldn't find file locally at /content/jdepoix/junit_test_completion/junit_test_completion.py, or remotely at https://huggingface.co/datasets/jdepoix/junit_test_completion/resolve/main/junit_test_completion.py. Please provide a valid dataset name.
Turns out the dataset should be committed to main branch. While the existing version seems to be in develop branch.

Thanks.

May I check with you is there any instruction or check-points available for reproducing

Dear ConTest Authors,

Congrats on the excellent work and my sincere appreciation to you for sharing this dataset with the community. May I check if there is any documentation/set-ups for reproducing your experimental results with the evaluation metric? Thanks again for your time and efforts.

Thanks & Regards
Weishi

Dataset Generation for Project: Missing Information

Firstly, thanks for sharing this artefact. I've been browsing through the repository and I've found your implementation and documentation to be quite valuable.

However, there seems to be a missing piece in the repo that I am particularly interested in: the data preprocessing and dataset creation. I wasn't able to find the code you used for mining GitHub repos or for mapping the focal method, focal class, etc. This is an integral part of the project for anyone looking to replicate your process or understand it more thoroughly.

I understand that these processes can be complex and might still be a work in progress. But, I believe even the incomplete scripts would be greatly beneficial for myself and others who are following this project. I would appreciate if you could share these parts of your work. Please let me know if you need any assistance in completing or cleaning up this part.

I am more than willing to contribute.