Comments (9)
no worry it will be done, it's quite easy for the mini-4k since it takes all llama2 arch.
fyi: https://forum.opennmt.net/t/phi-3-3-8b-llama2-7b-ensemble-just-for-fun/5729
from ctranslate2.
PR #1680 to add the converter for Phi3
from ctranslate2.
I second this. The current phi loader is broken, apparently because of some changes that Microsoft did to the model after it was initially released. At any rate, adapting the phi loader to the new phi3 should be easier than starting from scratch.
from ctranslate2.
For anyone else researching this, phi3 support has been added to the convert_hf_to_gguf.py script in llama.cpp. Perhaps something can be gleaned from there to simplify the implementation of the ct2 converter.
from ctranslate2.
Is it done yet? I've been waiting patiently for approximately two hours now? ;-)
from ctranslate2.
Hello, I am working on it. Some unexpected problems appears.
from ctranslate2.
I'm not skilled enough to help directly by implementing the code...but if you want me to do any grunt work or research let me know dude...anything to assist speed up the process. Thanks!
from ctranslate2.
I'd like to start learning to eventually possibly help...Question...how do I get the actual model architecture to start with...It's my understanding that getting the model's structure, what activation functions are used, etc. and basically starting to understanding the structure is key in making additional converters down the road. For example, here's a link:
Here are some other links that I've been gathering with the goal of eventually contributing a converter...based on first trying to understand the structure of LLMs...
https://github.com/mert-kurttutan/torchview
https://github.com/lutzroeder/netron
Huggingface sometimes (but not always) has information like this...
Basically, any good starting point for me that you'd recommend dude? Thanks!
from ctranslate2.
Remember, you're dealing with an idiot who doesn't do this for a profession and has never taken the LLM 101 class in college let alone have a doctoral degree. ;-) I don't even know what "mlp.down" or "layernorm.weight" means, for example, but am willing to learn.
from ctranslate2.
Related Issues (20)
- Dynamic LoRA switching HOT 1
- [SOLVED] Running Llama3 with Ctranslate2 HOT 4
- target_prefix latency HOT 2
- Unexpected inference results from Flan-T5 XXL converted to ctranslate2 with version 4.2.1 and 4.1.1 (using tensor parallel) HOT 4
- How to compile from source on windows 11? HOT 3
- Can't hide GPUs to get_cuda_device_count() HOT 5
- opus-mt-en-zh does not respect the end token
- I got invalid conversion error when compile on linux HOT 2
- CTranslate2 cmake error when trying to build the code from source with cuda support enabled on Windows. HOT 6
- libctranslate2-81fc0d88.so.4.2.1 in python package has executable stack flag
- Whisper encode roughly 4x slower than openai/pytorch HOT 1
- Option --self_attn_type scaled-dot-flash is not supported (supported values are: scaled-dot) HOT 6
- Doesn't build without docker. libiomp5 not found HOT 6
- Clang unusual switches wrongly hardcoded in resulting setup.py HOT 3
- Support for Phi3-Small, Medium, and Vision HOT 1
- Different results when run with tensor parallelism HOT 2
- CUDA DeviceAllocate segfault HOT 3
- Converter not working for NLLB models HOT 4
- Ctranslate2 Pypi exceeds limit 20GB
- Facing issues with Ctranslate2 when working with Intel built-in GPU and oneDNN HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from ctranslate2.