sileod / tasknet Goto Github PK
View Code? Open in Web Editor NEWEasy multi-task learning with HuggingFace Datasets and Trainer
License: GNU General Public License v3.0
Easy multi-task learning with HuggingFace Datasets and Trainer
License: GNU General Public License v3.0
Hi! I tried the basic 3-task example from the README file, and the training worked fine. Then I tried to save and load the model:
Saving the model worked ok:
trainer.save_model("tasknet-model")
But loading the model gives an error:
loaded = tn.Model.from_pretrained('./tasknet-model')
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
Cell In[16], line 1
----> 1 loaded = tn.Model.from_pretrained('./tasknet-model')
File ~/projects/keha/Tekoaly/trials/skillrecommendation-language-model/venv/lib/python3.10/site-packages/transformers/modeling_utils.py:2175, in PreTrainedModel.from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
2173 if not isinstance(config, PretrainedConfig):
2174 config_path = config if config is not None else pretrained_model_name_or_path
-> 2175 config, model_kwargs = cls.config_class.from_pretrained(
2176 config_path,
2177 cache_dir=cache_dir,
2178 return_unused_kwargs=True,
2179 force_download=force_download,
2180 resume_download=resume_download,
2181 proxies=proxies,
2182 local_files_only=local_files_only,
2183 use_auth_token=use_auth_token,
2184 revision=revision,
2185 subfolder=subfolder,
2186 _from_auto=from_auto_class,
2187 _from_pipeline=from_pipeline,
2188 **kwargs,
2189 )
2190 else:
2191 model_kwargs = kwargs
AttributeError: 'NoneType' object has no attribute 'from_pretrained'
I wonder what is the correct way to save and load the model?
Hi,
Using python 3.11 @dataclass
decorator throws an error. Here is the traceback
import tasknet as tn
/venv/lib/python3.11/site-packages/tasknet/__init__.py:1: in <module>
from .models import *
/venv/lib/python3.11/site-packages/tasknet/models.py:23: in <module>
from .tasks import Classification
/venv/lib/python3.11/site-packages/tasknet/tasks.py:198: in <module>
@dataclass
/usr/local/lib/python3.11/dataclasses.py:1230: in dataclass
return wrap(cls)
/usr/local/lib/python3.11/dataclasses.py:1220: in wrap
return _process_class(cls, init, repr, eq, order, unsafe_hash,
/usr/local/lib/python3.11/dataclasses.py:958: in _process_class
cls_fields.append(_get_field(cls, name, type, kw_only))
/usr/local/lib/python3.11/dataclasses.py:815: in _get_field
raise ValueError(f'mutable default {type(f.default)} for field '
E ValueError: mutable default <class 'tasknet.tasks.DataCollatorForMultipleChoice'> for field data_collator is not allowed: use default_factory
It looks like a field
would be needed here.
Currently this is not a parameter that we can pass to the TextClassificationPipeline
from the load_pipeline
function. It would be interesting to have the functionality when using the model on specific tasks for inference.
The class Adapter expects Z in constructor:
class Adapter(transformers.PreTrainedModel):
config_class = transformers.PretrainedConfig
def __init__(self, config, classifiers=None, Z=None, labels_list=[]):
super().__init__(config)
self.Z= torch.nn.Embedding(len(config.classifiers_size),config.hidden_size, max_norm=1.0).weight if Z==None else Z
self.classifiers=torch.nn.ModuleList(
[torch.nn.Linear(config.hidden_size,size) for size in config.classifiers_size]
) if classifiers==None else classifiers
self.config=self.config.from_dict(
{**self.config.to_dict(),
'labels_list':labels_list}
)
def adapt_model_to_task(self, model, task_name):
task_index=self.config.tasks.index(task_name)
#setattr(model,search_module(model,'linear',mode='class')[-1], self.classifiers[task_index])
model.classifier=self.classifiers[task_index]
return model
def _init_weights(*args):
pass
but doesn't use it at all when adapting model to task?
Do you have any example of how this work?
I tried to run this to calculate cola but it did not work:
https://huggingface.co/sileod/deberta-v3-base-tasksource-nli/discussions/2
At the moment we cannot specify a device when loading a pipeline using the load_pipeline
helper function. To have a fine grained control over how pipelines are loaded, it would be nice too pass it to the TextClassificationPipeline
constructor.
Hello, sorry im quite new to writing issues.
I trained a joint token classification and sequence classification model. To save it i used this:
trainer.save_model("multi_task/")
However trying to load the same model, i faced this issue
model_2 = tn.load_pipeline("/kaggle/working/multi_task","intent_classification", adapt_task_embedding=True)
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ in <module>:1 │
│ │
│ ❱ 1 model_2 = tn.load_pipeline("/kaggle/working/multi_task","intent_classification", adapt_ │
│ 2 │
│ │
│ /opt/conda/lib/python3.10/site-packages/tasknet/utils.py:198 in load_pipeline │
│ │
│ 195 │ │ import tasksource │
│ 196 │ except: │
│ 197 │ │ raise ImportError("Requires tasksource.\n pip install tasksource") │
│ ❱ 198 │ task = tasksource.load_task(task_name, multilingual=multilingual) │
│ 199 │ │
│ 200 │ model = AutoModelForSequenceClassification.from_pretrained( │
│ 201 │ │ model_name, ignore_mismatched_sizes=True │
│ │
│ /opt/conda/lib/python3.10/site-packages/tasksource/access.py:102 in load_task │
│ │
│ 99 │ query = dict_of(id, dataset_name, config_name, task_name,preprocessing_name) │
│ 100 │ query = {k:v for k,v in query.items() if v} │
│ 101 │ _tasks = (lmtasks if multilingual else tasks) │
│ ❱ 102 │ preprocessing = load_preprocessing(_tasks, **query) │
│ 103 │ dataset = load_dataset(preprocessing.dataset_name, preprocessing.config_name, **load │
│ 104 │ dataset= preprocessing(dataset,max_rows, max_rows_eval) │
│ 105 │ dataset.task_type = preprocessing.__class__.__name__ │
│ │
│ /opt/conda/lib/python3.10/site-packages/tasksource/access.py:90 in load_preprocessing │
│ │
│ 87 │
│ 88 def load_preprocessing(tasks=tasks, **kwargs): │
│ 89 │ _tasks_df = list_tasks(multilingual=tasks==lmtasks) │
│ ❱ 90 │ y = _tasks_df.copy().query(dict_to_query(**kwargs)).iloc[0] │
│ 91 │ preprocessing= copy.copy(getattr(tasks, y.preprocessing_name)) │
│ 92 │ for c in 'dataset_name','config_name': │
│ 93 │ │ if not isinstance(getattr(preprocessing,c), str): │
│ │
│ /opt/conda/lib/python3.10/site-packages/pandas/core/indexing.py:1073 in __getitem__ │
│ │
│ 1070 │ │ │ axis = self.axis or 0 │
│ 1071 │ │ │ │
│ 1072 │ │ │ maybe_callable = com.apply_if_callable(key, self.obj) │
│ ❱ 1073 │ │ │ return self._getitem_axis(maybe_callable, axis=axis) │
│ 1074 │ │
│ 1075 │ def _is_scalar_access(self, key: tuple): │
│ 1076 │ │ raise NotImplementedError() │
│ │
│ /opt/conda/lib/python3.10/site-packages/pandas/core/indexing.py:1625 in _getitem_axis │
│ │
│ 1622 │ │ │ │ raise TypeError("Cannot index by location index with a non-integer key") │
│ 1623 │ │ │ │
│ 1624 │ │ │ # validate the location │
│ ❱ 1625 │ │ │ self._validate_integer(key, axis) │
│ 1626 │ │ │ │
│ 1627 │ │ │ return self.obj._ixs(key, axis=axis) │
│ 1628 │
│ │
│ /opt/conda/lib/python3.10/site-packages/pandas/core/indexing.py:1557 in _validate_integer │
│ │
│ 1554 │ │ """ │
│ 1555 │ │ len_axis = len(self.obj._get_axis(axis)) │
│ 1556 │ │ if key >= len_axis or key < -len_axis: │
│ ❱ 1557 │ │ │ raise IndexError("single positional indexer is out-of-bounds") │
│ 1558 │ │
│ 1559 │ # ------------------------------------------------------------------- │
│ 1560 │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
IndexError: single positional indexer is out-of-bounds
I intend to load the model from this checkpoint for domain adaption
How do i address this issue? Also if you could redirect me to some resources to understand adapters that would be pretty helpful (i found out from the other issue posted here).
PS: Thank you for this incredible library- it saved me a lot of time
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.