Comments (2)
bghira
commented on August 9, 2024
4
pytorch 2.3 has bf16 and complex tensor support, and dockerised containers now work @lamroger
from gemma_pytorch.
lamroger
commented on August 9, 2024
Took a look, a few things.
Linux / mps support looks to be in progress still pytorch/pytorch#81224 so running in a container isn't ready yet.
MPS has some limitations around complex tensors atm. Since gemma uses RoPE, it uses complex tensors and errors out if you run it locally.
Line 426 in 01062c9
Running locally with python scripts/run.py --ckpt gemma-2b-it.ckpt --variant 2b --device mps
from gemma_pytorch.
Related Issues (20)
- Cannot run on v4-16 worker 0 TPU VM: "Failed to get global TPU topology" HOT 5
- After deplyed google/gemma-7b-it, there always is error response. HOT 8
- [Question] Embeddings normalization by sqrt(hidden_size) HOT 4
- is it possible to convert gemma_pytorch to onnx to tflite? HOT 2
- not found weight file HOT 3
- Inconsistency between PyTorch and JAX implementation HOT 2
- How to fine-tune Gemma with pytorch? HOT 2
- Objectivity
- Is max_position_embeddings=8096 neccessary in 2b model? HOT 3
- Can't disable sampling
- I got empty result while using 7b-it model HOT 4
- Error when running Gemma inference on GPU HOT 1
- Is it possible to load 7b-it using quantization config HOT 1
- Unable to reproduce MATH resulst HOT 2
- How to save memory when loading weights? HOT 1
- How to use gemma for multi-round conversations HOT 2
- Error when run docker/Dockerfile HOT 1
- Prepare model for deployment to Private Vertex AI endpoint HOT 5
- Gemma finetuning formatting
- when to support RecurrentGemma? HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from gemma_pytorch.