uf-hobi-informatics-lab/gatortron

all scripts used in gatortron project

gatortron's Issues

Medical Dataset

Hi, do you plan on releasing the dataset that you have built that includes all medical knowledge?

support int8 from huggingface?

ValueError: MegatronBertForSequenceClassification does not support device_map='auto' yet.
thanks!

Discrepancy between vocabulary size in model and tokenizer leading to bugs

Hi! Had a quick question about the discrepancy between the input embeddings:

model = AutoModel.from_pretrained('UFNLP/gatortron-base')
model.embeddings.word_embeddings.shape

There are 50176 in this module, but the tokenizer has 50101 vocabulary items (https://huggingface.co/UFNLP/gatortron-base/raw/main/vocab.txt).

Is there a reason for this discrepancy? It is making us hard-code the vocabulary size to fix this, and we hope we are correctly initializing from gatortron.

Otherwise, thank you so much for open sourcing this! It is extremely helpful :)

How many models can be generated?

Congratulations on this amazing article. I have a question about the fine-tuning process. Does each fine-tuning for each task generate a different model in the end? Or can we say that GatorTron is a multitask model?

How to use pretrained network for NER?

Thanks for sharing your work. How can I use the pretrained network for a downstream task such as NER? I am a beginner to LLMs and NVIDIA LLM frameworks. Would appreciate any help. Thanks!

how to run model and use it

Hello,
I need to know how to load the model and run it?
even though, by searching online, I could not find any useful resource.
I am new with NEMO.

thanks,

How to run model and finetune it

Hi Guys,
I need to know how to load the GatorTron model and run it. GatorTron exactly matches with my requirement and i was in searching online but i could't find any useful resources. Can you please help me on this model, Thanks in advanced.

Example of generating text from gatortron model?

Hi,

I'm new to huggingface and gatortron. I wish to generate text from a medical description for example, but not sure how to do it with the gatortron models. I've tried to adapt other examples (e.g. https://huggingface.co/docs/transformers/tasks/language_modeling ) to the gatortron model, but have not had much luck.

How can I use gatortron* for generating text?

from transformers import AutoTokenizer, AutoModelForCausalLM

prompt = "Somatic hypermutation allows the immune system to"

tokenizer = AutoTokenizer.from_pretrained('UFNLP/gatortronS')

inputs = tokenizer(prompt, return_tensors="pt").input_ids

model = AutoModelForCausalLM.from_pretrained('UFNLP/gatortronS', is_decoder=True)

outputs = model.generate(input_ids=inputs, max_new_tokens=100, do_sample=True, top_k=50, top_p=0.95, temperature=0.9)

tokenizer.batch_decode(outputs, skip_special_tokens=True)

I only get dots...

['somatic hypermutation allows the immune system to ....................................................................................................']

I've also tried the pipeline example:

prompt = "Somatic hypermutation allows the immune system to"
from transformers import pipeline

generator = pipeline("text-generation", model='UFNLP/gatortronS')
generator(prompt)

But it does not work either.

If you want to use `MegatronBertForCausalLM` as a standalone, add `is_decoder=True.`
Xformers is not installed correctly. If you want to use memory_efficient_attention to accelerate training use the following command to install Xformers
pip install xformers.
/home/user1/.local/lib/python3.9/site-packages/transformers/generation/utils.py:1369: UserWarning: Using `max_length`'s default (20) to control the generation length. This behaviour is deprecated and will be removed from the config in v5 of Transformers -- we recommend using `max_new_tokens` to control the maximum length of the generation.
  warnings.warn(

[{'generated_text': 'Somatic hypermutation allows the immune system to to to to to to to to to to to to to'}]

Thank you for your help and great project.