Comments (2)
Haijian06
commented on August 16, 2024
你好感谢肯定😊，是的微调的时候需要遵守Yi模型微调的格式，但现在其实不用这么复杂，你可以参考https://github.com/01-ai/Yi-1.5?tab=readme-ov-file#fine-tuning 里面的提及的微调框架，使用这些微调框架会自动转换成该模型需要的数据集格式。
from yi-1.5.
nuoma
commented on August 16, 2024
你好ryx103，
您提到的‘数据中的特殊tag’并不是必要的，实际上要看训练代码对于数据集是如何进行处理的。比如针对于'chosen'的处理：
https://github.com/01-ai/Yi/blob/11d140d641d18e1190caad2d5170da3b76c9e4f6/finetune/utils/data/raw_datasets.py#L24
当然了，我们提供的finetune并没有持续的维护。在现在这个时间点，很多的训练框架都更新/维护的非常好，比如 https://github.com/hiyouga/LLaMA-Factory 。一般来说，SFT数据集会采用ShareGPT格式是更加通用的做法。
希望这能解答你的疑惑 :-)
from yi-1.5.
Related Issues (20)
- 4K上下文完全不够用啊，能出个16K的吗？ HOT 3
- Quick start code HOT 1
- Inquiries about the AGIEval setup HOT 1
- 除了34B，其他小参数模型的指令跟随能力都不行 HOT 7
- modelscope模型下载问题 HOT 5
- Will Yi-large be published in open source? HOT 1
- 对于发展方向，提点小建议 HOT 1
- `max_length` (=20) to control the generation length. HOT 1
- 请问yi-large考虑登录一些第三方分发平台吗 HOT 1
- Yi-1.5-9B指标没法复现 HOT 1
- Does Yi-1.5-Chat model use the standard CHATML template? HOT 1
- 需要34b-chat-16k 量化版本 HOT 4
- test
- 可以请问一下yi-1.5-34b chat推理超参数吗，想复现在alignbench上的效果
- Fast tokenizer HOT 2
- 关于 tokenizer 编码 <|im_start|> 的问题 HOT 2
- tokenizer bug
- tokenizer的问题 HOT 2
- Fast Tokenizer add unexpected space token HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from yi-1.5.