Comments (5)
Hello, what would be the use case of a progress bar? If running it on a batch of text, you can get a quick progress bar by using tqdm in your for loop.
from detoxify.
That's in case I wanna query sentences one by one, which is not good because it slows the machine down as the model does initialization for each sentence.
The faster way would be to query an array of sentences. For that a progress-bar is useful.
from detoxify.
It shouldn't initialise the model for each sentence, are you defining the model first and then do the prediction for each sentence? e.g.
model = Detoxify("unbiased")
for batch in tqdm(data_batches):
results = model.predict(batch)
from detoxify.
IDK. I had seen in other hugging face models that each batch starts with a low speed and then the progress bar goes more fastly when it moves more to the end (for each batch).
Thus, I think if we only have one collective batch, the overall speed will be higher (?)
from detoxify.
Depends on how big your batch is and how much you can fit into your memory, if it's large enough it might be more efficient to do it in smaller batches. Are you doing this on cpu or gpu?
from detoxify.
Related Issues (20)
- Pinpoint the parts of the speech that trigger high values
- OSError: Unable to load vocabulary from file. Please check that the provided vocabulary is accessible and not corrupted. HOT 1
- Number of epochs to get the best model HOT 2
- TypeError: 'NoneType' object is not subscriptable HOT 8
- UnicodeDecodeError when installing from git
- Add dutch language HOT 1
- Question regards use case HOT 1
- Question - Adding additional models and labels. HOT 1
- Question regards training with other models HOT 1
- Error during training HOT 1
- Multi GPU predict
- Add license information for toxic-bert on HF HOT 7
- Converting model to AWS Inferentia hardware using Optimum-cli
- FileNotFoundError: [Errno 2] No such file or directory: 'jigsaw_data/jigsaw-toxic-comment-classification-challenge/val.csv'
- What are the max token lengths for the models?
- Error on PIP install HOT 6
- Bump up Transformers version HOT 2
- "torch" error by just importing Detoxify HOT 1
- OSError: None is not a local folder and is not a valid model identifier listed on 'https://huggingface.co/models' HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from detoxify.