mickmelon/pedgpt

AI-driven peds for GTA V/FiveM using GPT, enhancing gameplay with realistic, engaging autonomous agents on .NET.

PedGPT 🤖

---

🛠️ Very WIP

The project has only just begun, so it's not in a real usable state. Drastic changes are expected.

---

PedGPT is an ambitious open-source project that aims to create intelligent autonomous agents for peds (pedestrians) in Grand Theft Auto V (GTA V) using advanced language models like GPT-4, built on .NET. This project takes inspiration from langchain, AutoGPT, and the ReAct framework to create peds that can reason, act, and interact in realistic and engaging ways.

The primary goal of this project is to enhance the gaming experience in GTA V by introducing AI-driven peds that possess more complex decision-making and behavior patterns compared to the original game NPCs. The integration with GTA V will be done through a FiveM server via gRPC.

Work has begun to move on over to using Semantic Kernel.

Overall process:

Signal processing:

Deep reasoning:

Getting started

⚠️There is no current limit or real measure to token usage, so be careful!

1. Clone this repository to your local machine:

git clone https://github.com/MickMelon/PedGPT.git

2. Navigate to the IntelliPed.ConsoleApp project folder:

cd src/IntelliPed.ConsoleApp

3. Set your OpenAI API key to your dotnet user secrets by running the following command:

dotnet user-secrets set "OpenAi:ApiKey" "{your api key}"
dotnet user-secrets set "OpenAi:OrgId" "{your org id}"

4. Build the solution using the following command:

dotnet build

5. Start PedGPT by running the following command:

dotnet run

Contribute

Contributions are welcomed to help improve PedGPT. To contribute:

1. Fork the repository.
2. Create a new branch for your feature or bugfix.
3. Make changes and commit them to your branch.
4. Submit a pull request, detailing the changes you made and why.

Licence

PedGPT is released under the GNU GPLv3 Licence.

pedgpt's Issues

Test with semantic kernel framework

Hi dude.
First, good job with this project. I was looking for something write in C# to start testing and found your project. I cloned the repo and I started to integrate semantic kernel : https://learn.microsoft.com/en-us/semantic-kernel/whatissk cause I think can be integrate as part of your main logic without risks. SK skills for example, can be integrated inside commands to decouple from any external logic.

Thanks again

What is the state of the project?

It looks to be very interesting to play with.

Is there a roadmap or something? I would like to know the current status of the project and potential milestones.

Feature req: Please integrate apipie.ai

Users want access to as much AI as they can get, they dont want to manage 50 accounts, they want the fastest AI they want the cheapest AI, and you can provide all of that for them with this update.

in addition to or in place of integrating with any aggregators - Please integrate APIpie so devs can access them all from one place/subscription and plus it also provides:

-The most affordable, reliable and fastest AI available
-One API to access ~500 Models and growing
-Language, embedding, voice, image, vision and more
-Global AI load balancing, route queries based on price or latency
-Redundancy for major models providing the greatest up time possible
-Global reporting of AI availability, pricing and performance

Its the same API format as openai, just change the domain name and your API key and enjoy a plethora of models without changing any of your code other than how you handle the models list.

This is a win win for everyone, any new AI's from any providers will be automatically integrated into your stack with this one integration. Not to mention all the other advantages.

Add Command prerequisites

Commands such as DriveToPosition should not be available for the agent when they are not driving a vehicle. This would perhaps be dependent on the agent's state. So, I think it would be good if command prerequisites could be defined.

I'm thinking of using attributes, something like this:

[CommandDescription("drive_to_position", "Drives to the specified coordinates.")]
[RequireState("Current vehicle", RequireStateCondition.NotEmpty)]
public record DriveToPositionCommand(float X, float Y, float Z) : ICommand
{
    public Task<CommandResult> Execute()
    {
        return Task.FromResult(new CommandResult(true, $"You have driven to '{X}, {Y}, {Z}'"));
    }
}

Not married to the RequireStateCondition bit, just an example. Would need some thought on how best to design this.

Then in the Observe method on the Agent, the commands that the agent can execute would be updated.

Add .NET Framework Projects to Validation Pipeline

Currently, the validation pipeline only builds the main IntelliPed projects, which are using .NET 8. There were difficulties building the FiveM projects, which use .NET Framework.

Decided it wasn't worth the time to investigate and resolve right now though, so created this issue to come back to it later.

JSON fixing

Often times, the agent will return a response that contains invalid JSON, or text before the JSON.

For example:

"Response:\r\n{\r\n    \"Thoughts\": {\r\n        \"Text\": \"I really feel like part of the party now that I\\u0027m dancing like a pro. This is a great way to blend in with the crowd and enjoy the music. But I can feel myself getting tired and thirsty, so I need to take a break and get hydrated.\",\r\n        \"Reasoning\": \"Dancing is a great way to fit in with the crowd, but I need to be mindful of my health and not overdo it.\",\r\n        \"Plan\": \"- Take a break soon and grab some water to stay hydrated.\\n- Dance some more and enjoy the music while staying aware of my surroundings and avoiding any dangerous or illegal activities.\",\r\n        \"Criticism\": \"I should have brought a water bottle with me to make it easier to stay hydrated. I also need to be careful not to stay in one spot for too long and become an easy target for any troublemakers.\",\r\n        \"Speak\": \"I\\u0027m having a great time dancing at Grove Street, but I need to take a break and grab some water soon. \"\r\n    },\r\n    \"Command\": {\r\n        \"Name\": \"dance\",\r\n        \"Args\": {}\r\n    }\r\n}"

There's currently some basic fixing in place in the JsonFixer class, however it may not fix all occurrences.

We should come up with a set of test data and write some unit tests for various invalid JSONs that the agent could respond with.