macoron/gpt4all.unity

Bindings of gpt4all language models for Unity3d running on your local machine

gpt4all.unity's Issues

String conversion error: Partial byte sequence encountered in the input and questions

Hi ! Sorry to bother you but i am a very beginner in unity ! ^^ I need your help about that error and GPU use in unity :

• I cloned the repository in my project.
• So i end up with 2 folders in my Asset folder named : "Samples" and "StreamingAssets"
• Until there. All was good.
• I have installed Gpt4all on my computer with the "installer for windows" downloaded on their website.
• I cloned llama.ccp repository and used w64devkit to build it.
• I downloaded ggml-gpt4all-j-v1.3-groovy.bin and placed it in StreamingAssets\Gpt4All
• I launched the unity project to see what happen and all worked.
• But when the chat moves towards the "enter text" in unity, the LLM stops writing its answer in mid-sentence.
• And I get this error :

ExecutionEngineException: String conversion error: Partial byte sequence encountered in the input.
(wrapper native-to-managed) Gpt4All.LlmWrapper.ResponseCallbackStatic(int,intptr)
Gpt4All.LlmWrapper.Prompt (System.String promptText, Gpt4All.LlmPromptContext context) (at Packages/com.gpt4all.unity/Scripts/LlmWrapper.cs:85)
Gpt4All.LlmWrapper+<>c__DisplayClass17_0.b__0 () (at Packages/com.gpt4all.unity/Scripts/LlmWrapper.cs:99)
System.Threading.Tasks.Task`1[TResult].InnerInvoke () (at :0)
System.Threading.Tasks.Task.Execute () (at :0)
--- End of stack trace from previous location where exception was thrown ---
Gpt4All.LlmWrapper.PromptAsync (System.String promptText, Gpt4All.LlmPromptContext context) (at Packages/com.gpt4all.unity/Scripts/LlmWrapper.cs:100)
Gpt4All.LlmManager.Prompt (System.String rawPrompt) (at Packages/com.gpt4all.unity/Scripts/LlmManager.cs:133)
Gpt4All.Samples.ChatSample.SendToChat (System.String prompt) (at Assets/Samples/1 - Chat/ChatSample.cs:43)
System.Runtime.CompilerServices.AsyncMethodBuilderCore+<>c.b__7_0 (System.Object state) (at :0)
UnityEngine.UnitySynchronizationContext+WorkRequest.Invoke () (at :0)
UnityEngine.UnitySynchronizationContext.Exec () (at :0)
UnityEngine.UnitySynchronizationContext.ExecuteTasks () (at :0)

Do you have an idea of what it is ?
Did I download and install things correctly ?
And finally, do you know how to use GPU instead of CPU ?

Thanks in advance Macoron !

Error: libllama.dll and libllmodel.dll

I encountered an issue when attempting to add a repository as a Unity Package to my existing project using the Git URL in Unity Package Manager. The error logs indicate that two DLL files, namely libllama.dll and libllmodel.dll, along with their dependencies, could not be loaded.

Steps to Reproduce:
1. Open Unity project.
2. Go to Unity Package Manager.
3. Click on the "+" button and select "Add package from git URL".
4. Enter the Git URL for the repository and proceed.
5. Wait for the package to be imported.
6. Observe the error message related to the DLL files and their dependencies not being loaded.

Error Logs:

• ../Library/PackageCache/com.gpt4all.unity@548076b369/Plugins/Windows/libllama.dll' because one or more of its dependencies could not be loaded.

• ../Library/PackageCache/com.gpt4all.unity@548076b369/Plugins/Windows/libllmodel.dll' because one or more of its dependencies could not be loaded.

Unity3D 2022.2.2f1
Win11

Thanks.

Is it possible to have several LLM sessions open at once?

Can I create multiple instances of LLM Manager to have several "chats" with distinct context and history existing simultaneously?

Failed to load 'libllama.dll' because one or more of its dependencies could not be loaded.

Could you fix this since the old dll in the commit that did work are now severely out dated?

** modelPath failing in LlmWrapper fix! **

**Sinds u promote both methods of installing (package manager or cloning/downloading the git)

**add this above async void Awake in LlManager.cs

    // Singleton instance
    public static LlmManager Instance { get; private set; }

**add this inside async void Awake in LlManager.cs

   if (Instance == null)
        {
            Instance = this;
        }
   else
       {
            DestroyImmediate(this);
        }

Put this method in Llmanager.cs:

    public string GetModelPath(string modelPath)
    {
        return Path.Combine(Application.streamingAssetsPath, modelPath);
    }

And this around line 117:

      // correcting faulty path by getting it from one central In-App location
      var path = GetModelPath(modelPath);

Then in LlWrapper.cs around line 202:

        // correcting faulty path by getting it from one central In-App location
        var path = LlmManager.Instance.GetModelPath(modelPath);

Crash during project loading

I've been reported that project may crash during opening in Unity. Still researching why it could be the case, but if you are experiencing something similar, try to checkout this commit:

67aeaab

Please report in this issue if you have this problem and if checkout to other commit helped you.

Doesn't work with 4bit?

Tried using with several ggml bins (some from TheBloke, some from GPT4All) and it seems like this code only works with models not labeled as 4bit? Or am I missing something?

Whisper not loaded and failed to load LLM

Hi Macoron !!!!!!!!!

It seems i found new errors in unity 😢 and this time, I took screenshots 😁 ! I tried to launch the project in a new unity project this morning and poof 😮! Here come the errors ☹️ ! Take a look at that :

AI respond to the instructions with every prompt

When filling out a larger instruction, the AI starts to pay less attention to the prompt and responds primarily with the instruction in mind.

I tried to fix this by removing the formation of the print input and instead giving it an initial print in init. This worked with smaller instructions but when given a longer instruction it refused to respond.

Feature Can you please make it for other platforms

Hi
Can you make it for Android, ios and webgl. If it's works for all platforms it's more useful.

C# bindings

Is it possible to use these DLLs (libllama.dll, libllmodel.dll) that in your project for a Windows console application using Target Framework .NET 6 or .NET 7 to run gpt4all?

Error loading model

hey getting this error please help

NullReferenceException: Object reference not set to an instance of an object
Gpt4All.LlmManager.InitModel () (at ./Packages/com.gpt4all.unity/Scripts/LlmManager.cs:114)
UnityEngine.Debug:LogException(Exception)
Gpt4All.d__37:MoveNext() (at ./Packages/com.gpt4all.unity/Scripts/LlmManager.cs:121)
System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1:SetResult(LlmWrapper)
Gpt4All.d__28:MoveNext() (at ./Packages/com.gpt4all.unity/Scripts/LlmWrapper.cs:214)
UnityEngine.UnitySynchronizationContext:ExecuteTasks()

Request for Simplified Response Format

Problem:
Currently, when interacting with gpt4all, the responses are generated in a conversational format, with character-by-character animated iterations. While this format can be useful for some applications, there are cases where a more simplified response format is desired.

Proposal:
I suggest implementing an option to receive responses from gpt4all as a single, complete text without character iteration. This would make it easier for developers to integrate gpt4all into applications where an animated response is not necessary or preferred.

Thanks.

Support for falcon models

Hello :)
Really nice repo!

https://docs.gpt4all.io/
Falcon transformer architecture seems to be supported by gpt4all is it possible to add it to the library?

problema with dll

I Clone the repository to be able to use it, downloaded the models "ggml-mpt-7b-chat" and "ggml-gpt4all-j-v1.3-groovy", I'm using the default one and it seems to take it well, but then it gives me the following error:

DllNotFoundException: libllmodel assembly:<unknown assembly> type:<unknown type> member:(null) Gpt4All.LlmWrapper.InitFromPath (Gpt4All.LlmModelType type, System.String modelPath) (at ./Packages/com.gpt4all.unity/Scripts/LlmWrapper.cs:161) Gpt4All.LlmWrapper+<>c__DisplayClass25_0.<InitFromPathAsync>b__0 () (at ./Packages/com.gpt4all.unity/Scripts/LlmWrapper.cs:198) System.Threading.Tasks.Task1[TResult].InnerInvoke () (at <5d4cbfbeb62e454f98e19b231866113e>:0)
System.Threading.Tasks.Task.Execute () (at <5d4cbfbeb62e454f98e19b231866113e>:0)
--- End of stack trace from previous location where exception was thrown ---
Gpt4All.LlmWrapper.InitFromPathAsync (Gpt4All.LlmModelType type, System.String modelPath) (at ./Packages/com.gpt4all.unity/Scripts/LlmWrapper.cs:199)
Gpt4All.LlmManager.InitModel () (at ./Packages/com.gpt4all.unity/Scripts/LlmManager.cs:111)
UnityEngine.Debug:LogException(Exception)
Gpt4All.d__34:MoveNext() (at ./Packages/com.gpt4all.unity/Scripts/LlmManager.cs:119)
System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1:SetException(Exception)
Gpt4All.d__25:MoveNext() (at ./Packages/com.gpt4all.unity/Scripts/LlmWrapper.cs:199)
UnityEngine.UnitySynchronizationContext:ExecuteTasks()

This message appears in the console when I hit play in the editor. I'm not sure if I'm missing a step or doing something wrong. I'm using Unity version 2022.2.9f1. I appreciate any information you can provide.
`