Comments (4)
thomwolf
commented on May 1, 2024
Yes, I hard coded that since the pre-trained models are all trained with gelu anyway.
from transformers.
lukovnikov
commented on May 1, 2024
ok. but since config is there anyway, isn't it cleaner to use it (to avoid errors for people using configs that use a different activation for some reason) ?
from transformers.
thomwolf
commented on May 1, 2024
Yes we can, I'll change that in the coming first release (unless you would like to submit a PR which I would be happy to merge).
from transformers.
lukovnikov
commented on May 1, 2024
yeah let me clean up and I'll PR
from transformers.
Related Issues (20)
- `device_map='auto' support for BertForSequenceClassification for inference using large models HOT 1
- WhisperForAudioClassification cannot evaluate during training using use_weighted_layer_sum HOT 3
- Loss calculation bug in BartForCausalLM
- Unused or unrecognized kwargs: padding, max_length, truncation. HOT 7
- Can Transformers AutoModel Support to(dml) HOT 2
- Layer_drop interferes with gradient_checkpoint operation in DDP. HOT 1
- Image classification pipeline fails with base64.encodebytes() input HOT 2
- llama-2 device_map (2,3) & `model.generate` HOT 4
- Possible issue with safetensors loading for TF models HOT 5
- Training model constantly increases memory consumption HOT 2
- What is the default multi-GPU training type? HOT 5
- Inconsitent module names (state_dict keys). HOT 6
- LayoutLM.from_pretrained doesn't load embeddings' weights when using safetensors HOT 1
- [Bug] Owlv2 Zero-Shot Object Detection HOT 5
- push_to_hub doesn't push checkpoint folder while training HOT 7
- AutomaticSpeechRecognition pipeline cannot predict WORD timestamps for Whisper models finetuned without timestamps prediction HOT 2
- `config_class` inconsistency for a custom model HOT 3
- Question about Multi-GPU inference (greedy generation with stopping criteria) HOT 2
- [BUG] pipeline raises ValueError with DeepSpeed HOT 3
- Question about how to write code for trainer and dataset for multi-gpu HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from transformers.