Comments (3)
I added a limit flag for the two examples you mentioned 08e8e3b
Run the examples again but with --limit 100
or any number less than the total number of samples.
As for the integrated intel CPUs/GPUs, I added a commit to allowing hooking into the fit/training loop. So you could do something like this:
# Snippet from biencoder finetuning
import intel_extension_for_pytorch as ipex
def ipex_hook(self, **kwargs):
param_optimizer = list(loss_model.named_parameters())
no_decay = ['bias', 'LayerNorm.bias', 'LayerNorm.weight']
optimizer_grouped_parameters = [
{'params': [p for n, p in param_optimizer if not any(nd in n for nd in no_decay)], 'weight_decay': 0.01},
{'params': [p for n, p in param_optimizer if any(nd in n for nd in no_decay)], 'weight_decay': 0.0}
]
optimizer = optimizer_class()
optimizer = kwargs['optimizer_class'](optimizer_grouped_parameters, lr = 2e-5)
model, optimizer = ipex.optimize(self.model, optimizer=optimizer)
self.model = model
# Wrapper needed as Sentence Transformer backend library will attempt to create it's own optimizer
def optimizer_wrapper(*args, **kwargs):
return optimizer
trainer = SentenceTransformerTrainer(
# Reuse the dataset with lazy static, so we don't have to do the preprocessing repeatedly
dataset=load_dataset(limit),
hparams_config=HparamConfig.from_json("hparam_cfg.json"),
evaluator_fn=evaluation.BinaryClassificationEvaluator,
)
# Overload with our prehook
trainer.prehook = ipex_hook
trainer.fit()
from debeir.
I think I can potentially package an index prebuilt for download. Also, which example are you referring to?
from debeir.
I think it was both finetune_biencoder and hparam_tuning that estimated around 4 hours or slightly more of runtime.
I am not sure if integrated Intel GPUs are supported by https://www.intel.com/content/www/us/en/developer/tools/oneapi/optimization-for-pytorch.html but if yes that could be worth investigating as well.
from debeir.
Related Issues (13)
- errors on build_test_env.sh HOT 1
- virtualenv vs venv in installation instructions HOT 1
- please add community guidelines HOT 1
- please add DeBEIR to PyPI HOT 1
- HTML documentation starting page about NIR, not DeBEIR HOT 1
- API Documentation incomplete HOT 4
- error with hparam tuning example HOT 2
- error with create_semantic_index.py example HOT 2
- errors with trec2022 examples HOT 2
- add README.md in pipline folder HOT 1
- upgrade torch major version to 2 HOT 2
- support Intel CPUs/GPUs by default
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from debeir.