yhydhx/chatgpt-api

Implements ChatGPT API via request package.

chatgpt-api's Issues

Did you obtain permission to use ChatGPT with n2c2 datasets?

The data use agreement forbids sharing the data in the manner done by your preprint without explicit permission:

5. Data User understands and agrees that the Data / Datasets are proprietary and confidential to Partners and agrees that Data User will not disclose, disseminate, or otherwise share the Data / Datasets to or with any other person or entity, including any subcontractor, for any purpose, without the prior written consent of Partners. To the extent Partners agrees in writing to permit such further access, the Data User will ensure that such further recipient of the Data / Datasets agrees in writing to all of the same restrictions, conditions and obligations that apply to Data User with respect to the Data / Datasets, and will make Partners a third-party beneficiary of such agreement.

Did you obtain such permission, and if so, were there any conditions on it?

Hi! 2014 i2b2/UTHealth de-identification challenge dataset Can not be accessed！

Could you send me a copy of data for testing and learning? Thanks! my email is [email protected]

If the code you provided was improved to include a specific model, how would one go about doing that?

I'm going to go out on a limb and ask, the code you provided doesn't have a specific model, it simply calls the ChatGPT API interface for the task, right?

Hello, I have some questions about this research?

Hello, dear authors

I have read your paper "DeID-GPT: Zero-shot Medical Text De-Identification by GPT-4"
https://arxiv.org/html/2303.11032v2

I would like to ask how to use the code you provided. Is it designed for preliminary de-identification, replacing anonymous text segments with similar categories of text?

Can this be tested on training data mixed with Chinese and English?

Could you provide an example of the prompt you used to ask ChatGPT?

Thank you.

"Accuracy" is recall

Hello!

In the preprint, the accuracy is calculated as a function of the confusion matrix entries $a = \frac{TP + TN}{TP + TN + FP + FN}$

However, in process_xml_public.py where the metrics are computed, the "accuracy" is found as the inverse of the fraction of annotated text which also appears in the deidentified versions:

    print("Remaining number of strings and Accuracy: ", sum, 1 - (sum/total_length))

which is not accuracy, but the recall metric $r = \frac{TP}{TP + FN}$ - you can attain perfect recall by simply removing everything from each document, which is then reported as "accuracy 1.0"

Do you compute the precision in addition to this, or do you have a way of estimating false positives/unnecessary redactions?