kleincode / llama2.jl Goto Github PK
View Code? Open in Web Editor NEWLlama 2 Inference in Julia. Developed for the JuliaML course at TU Berlin.
License: MIT License
Llama 2 Inference in Julia. Developed for the JuliaML course at TU Berlin.
License: MIT License
lines 367-409 in llama2.c
Maybe test small valued vectors on math functions. Especially with multiplication.
Maybe even try to brute force bad behaviour.
A parameter list for the individual functions would be helpful.
F.E.
rmsnorm
x [point in space]
weight [weight for multiplication]
Real dummy example, but with a more complex function that does not do some obvious math it might be helpful.
lines 616-665
Depends on #2
document that it's necessary to download the weights or better yet do that automatically when installing the package.
add documentation how set up and use the package
also a example would be nice
I saw that you have weight download functionalities in your test.
https://github.com/kleincode/Llama2.jl/blob/main/test/utils.jl
Would be nice to have this feature also as part of your main package. Would be much easier to get started here:
https://kleincode.github.io/Llama2.jl/dev/example/#Getting-Started
lines 445-571
Depends on #4
lines 577-614
lines 418-429 in llama2.c
Depends on #4
lines 111-162 in llama2.c
Depends on #3
Hi all,
here's some quick feedback before the second code review session next week.
I'm opening this as a single issue to not spam your repository, which already looks really good!
Tokenizer
struct and Tokenizer
constructor in separate places. Use a single docstring for both and document the way you expect users to use your struct.Line 10 in a4c3241
Line 48 in a4c3241
Float32
. I understand that this makes sense for a direct port of the C code base. However, in general, a parametric type Sampler{T}
would make sense here.Lines 20 to 23 in a4c3241
Line 32 in a4c3241
AbstractArray
, not just Vector{Float32}
softmax
function isn't compatible with ForwardDiff.jl's forward mode AD, since that uses a custom dual number type, which you prohibit.Could you provide more code examples for individual components? Especially with ML libraries they get really complex.
I noticed that you do not use keyword arguments (https://docs.julialang.org/en/v1/manual/functions/#Keyword-Arguments) for e.g. the predict function. It would be nice if you could run e.g.:
generate(transformer, tokenizer, sampler, testprompt, max_steps=10)
instead of how it is currently:
generate(transformer, tokenizer, sampler, testprompt, false, true, false, 10)
PS: Also as discussed, max_steps is not working currently...
mul!(out, in1, in2)
@view
where applicable, maybe even an @views
in front of the whole forward function? But make sure to copy token_embedding_table
into x
in line 144!rmsnorm
, softmax
, swiglu
with in-place versionsmutable
from mutable struct RunState
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.