Comments (10)
A model card would make sense. If there weren't any new per-say techniques then I wouldn't see the need for yet another paper. If there is, then sure!
from grok-1.
Is there a scientific paper accompanying this release? I've searched but couldn't find one. I find it odd that the weights would be released but not the research.
Because there's no research underlying it? Nothing new or surprising in the model so far, it's just the same architecture as other MoE LLMs with different data and training compute.
Not every software needs a paper that's going to be rejected at conferences and stay at pre-print stage on ArXiV. :)
from grok-1.
A model card would make sense. If there weren't any new per-say techniques then I wouldn't see the need for yet another paper. If there is, then sure!
Agree that model card (what data Grok was trained on) is crucial for this being truly "open source"
It looks like Grok had a model card back from last November: https://x.ai/model-card/
Training data The training data used for the release version of Grok-1 comes from both the Internet up to Q3 2023 and the data provided by our AI Tutors.
I doubt we'll get an answer any more detailed than "the Internet" and "whatever synthetic data our employees made"
from grok-1.
That's a technical report at best though.
Call it what you want, the issue is that it doesn't exist.
from grok-1.
Absolutely needs some experiment details on μTransfer of a MOE model that large, if someone noticed several 'weird' multiplier here.
Lines 31 to 47 in 7050ed2
from grok-1.
It would be nice to have:
- Paper explaining the methodology
- Benchmarks
- Data this was trained on
from grok-1.
Is there a scientific paper accompanying this release? I've searched but couldn't find one. I find it odd that the weights would be released but not the research.
Because there's no research underlying it? Nothing new or surprising in the model so far, it's just the same architecture as other MoE LLMs with different data and training compute. Not every software needs a paper that's going to be rejected at conferences and stay at pre-print stage on ArXiV. :)
Disagree. I think @Explosion-Scratch did a good job pointing out why a paper would be useful in this case.
from grok-1.
I'm happy as long as the code is up to date and the science is released even if not in an academic setting.
from grok-1.
That's a technical report at best though.
from grok-1.
from grok-1.
Related Issues (20)
- deleter of toke.
- Grok gt
- Grok AI
- I would like to run grok-1 locally using ollama but I cannot find prompt information to build out a Modelfile
- Share a Docker Image of Grok-1 HOT 5
- 0x331a94dac38795536d418a18cbaaf315d3188684
- Grok-1.5V code release HOT 4
- Related to run it locally HOT 2
- I have 10 A100 gpu, but 4 in one host, 6 in another host, how to install grok loacally?
- Having trouble with run.py (UPDATED, RESOLVED) (Maybe don't try run.py on one GPU) HOT 2
- jax-releases dependencies only for cuda* for linux
- How much memory is "enough"? HOT 5
- Идеальная девушка
- TypeError: dynamic_update_slice update shape must be smaller than operand shape, got update shape (1,) for operand shape (0,) HOT 1
- Can someone add a api
- grok open release ? it means just release open? and core is closedAI?
- Can someone tag a verson?
- SaveTF2
- xAI grok-1 GPU vram requirements (and same for cpu / dram)
- A release versoin tag?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from grok-1.