pzelasko/daseg

Dialog Acts SEGmentation: Tools for dialog act research

daseg's Issues

Inference at the offline (no internet connection)

Hi again :) I get this error if I try to make inference with no internet connection, looks like the problem is with the tokenizer, any clue about where to download the config.json and what path i have to change?
**json.load(open(Path(model_path) / 'tokenizer_config.json'))
NotADirectoryError: [Errno 20] Not a directory: 'DA_model/checkpointepoch=8.ckpt/tokenizer_config.json'

NotADirectoryError during evaluate the model with "dasg evaluate './exp-swda-longformer-512/checkpointepoch=6.ckpt' --device cuda -b 1 -o exp-swda-longformer-512/results.pkl -v"

Hi, I tried to run model as the "Running the experiments" in readme.md
the model trained successfully and saved the best checkpoint(epoch 6), but when i evaluate on the test set, I got the following error, may I know how to solve it?

Traceback (most recent call last):
File "/Work/daseg/daseg/models/transformer_model.py", line 34, in from_path
return TransformerModel.from_pl_checkpoint(model_path, device)
File "/Work/daseg/daseg/models/transformer_model.py", line 76, in from_pl_checkpoint
del ckpt['state_dict'][k]
KeyError: 'model.longformer.pooler.dense.weight'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/anaconda3/envs/daseg/bin/dasg", line 7, in
exec(compile(f.read(), file, 'exec'))
File "/Work/daseg/daseg/bin/dasg", line 351, in
cli()
File "/home/anaconda3/envs/daseg/lib/python3.8/site-packages/click/core.py", line 1128, in call
return self.main(*args, **kwargs)
File "/home/anaconda3/envs/daseg/lib/python3.8/site-packages/click/core.py", line 1053, in main
rv = self.invoke(ctx)
File "/home/anaconda3/envs/daseg/lib/python3.8/site-packages/click/core.py", line 1659, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/anaconda3/envs/daseg/lib/python3.8/site-packages/click/core.py", line 1395, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/anaconda3/envs/daseg/lib/python3.8/site-packages/click/core.py", line 754, in invoke
return __callback(*args, **kwargs)
File "/Work/daseg/daseg/bin/dasg", line 73, in evaluate
model = TransformerModel.from_path(Path(model_path), device=device)
File "/Work/daseg/daseg/models/transformer_model.py", line 46, in from_path
**json.load(open(Path(model_path) / 'tokenizer_config.json'))
NotADirectoryError: [Errno 20] Not a directory: 'exp-swda-longformer-512/checkpointepoch=6.ckpt/tokenizer_config.json'

Provide a trained checkpoint

Can you please provide a trained checkpoint to download and use just for inference without the need to train the model.

Advice on training with custom data?

Hi there Piotr,

I'm exploring your paper/approach for a custom dialog dataset as opposed to SWDA and MRDA. At first glance, your code appears quite tightly coupled with those two datasets. The dialog data that I'd like to test on is pretty simple - it has Speaker, Utterance, and a simple Label - without any of the other metadata present in SWDA. Can you please advise on the best approach for using your code on simpler data? I get lost figuring out how DialogActCorpus.from_swda_path produces a TensorDataset. Could you provide some advice regarding what would be sufficient for producing a TensorDataset for a dataset as described?

Thanks!

Simple Inference from a transcript

First of all thank you for your awesome work.. second I am asking if there is a sample example for doing a simple inference
input the full transcript and outputs the DA boundaries and Classes?
All I found here in the command line examples was for doing evaluation which needs the entire datasets.
Thanks in advance.

Error while inference

Hello again, I faced this error when the input text exceeded certain number of characters (15987) to be exact, or about 3k words approximately
" return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)
IndexError: index out of range in self "
form my search I found that this error means that I got an index out of the embed-dings range how could his happen? I checked the token and it was normal letters so it should not be OOV or something? any idea about what is going on here and how to fix it? thanks in advance.