microsoft/promptcraft-robotics

Community for applying LLMs to robotics and a robot simulator with ChatGPT integration

promptcraft-robotics's Issues

Multiple drones with ChatGPT

Hi everyone, first time postting

I´m interested in the application of chatGPT for robotics. I´m following the instrucctions to run a ChatGPT-AirSim Interface. But I want to go a step further and I want to controll multiple drones simultaniously.

I have been modifying the files related to the "AirSimInspection" simulation: settings.json, chatgpt_airsim.py and airsim_wrapper. I will attach the code as a .txt for everyone to see.

I can create multiple drones on the simulation and chatGPT its ackonwledging this on the command window. But the drones are not moving and I´m running out of ideas on how to solve this. Anyone has any idea, please?

Thanks for your suport.
airsim_wrapper.txt
chatgpt_airsim.txt
settings.txt

Report on Imminent Threats Posed by Drone Swarms. A Call for Urgent Action

Report on Imminent Threats Posed by Drone Swarms. A Call for Urgent Action
Executive Summary
The rapid advancement of drone technology has opened up new possibilities for various applications, ranging from agriculture and surveillance to emergency response. However, the same technology also poses unprecedented threats to public safety and national security. This report aims to highlight the imminent risks associated with drone swarms and the current lack of effective countermeasures.
Introduction
Drones have become increasingly sophisticated, capable of autonomous flight, and can be deployed en masse as swarms. While their potential for good is undeniable, the ease with which they can be misused is alarming. The ethical implications are particularly concerning, given the indiscriminate use of drones in warfare and the precedent it sets for malicious actors.
The Dilemma
Ontological Concerns
The ethical nature of drone technology and its potential for harm is a significant concern. The indiscriminate use of drones in warfare has set a dangerous precedent that could be exploited by malicious actors, both state and non-state.
Epistemic Concerns
The technology's capabilities and limitations are not fully understood. Traditional defense mechanisms are ill-equipped to handle such unconventional and rapidly evolving threats.
Imminent Threats

1. Ease of Deployment: Drones can be easily deployed from various platforms, including semi-trucks, making them highly mobile and difficult to trace.
2. Mass Attacks: Drone swarms can act as multiple active shooters, attacking simultaneously with little to no warning.
3. Escalation: Current countermeasures like ECM, hacking, nets, and no-fly zones are ineffective in the face of more robust and advanced drone swarms.
4. Unpredictability: The autonomous nature of drone swarms makes them highly unpredictable, further complicating defense strategies.
   Lack of Viable Solutions
5. EMP: Requires a lot of power and has localized effects.
6. Microwaves: Effective only until escalation leads to more robust swarms.
7. Local Monitoring and Shelters: Limited in scope and effectiveness.
8. Detection: Current detection methods are inadequate, and escalation would render them ineffective.
   Conclusion and Recommendations
   The lack of effective countermeasures against drone swarms presents an imminent threat to public safety and national security. Given the ethical implications and the limitations of current defense mechanisms, there is an urgent need for multi-disciplinary research and development to find viable solutions.
   Immediate actions must include:
9. Ethical Reevaluation: A thorough ethical review of drone technology and its applications.
10. Research and Development: Investment in R&D for effective countermeasures against drone swarms.
11. Public Awareness: Educating the public about the risks associated with drone technology.
12. Policy and Regulation: Implementing stringent policies and regulations to control the use and deployment of drones.
    This situation calls for a reevaluation of our actions and beliefs to ensure alignment with core principles. It serves as a grim reminder that technological advancements can be a double-edged sword, and ethical considerations must be at the forefront of any solutions we seek.

I was wondering about the 'def ask' part of the code

Hi, I am currently using the provided code and testing it on my drone for a personal project. I am using it to perform very simple movements such as "go up 40cm then land". So for my case, will it be better if I get rid of the appending part from 'def ask'? Because I won't be needing past conversations from the previous questions. I think appending the chat_history is making it take more time and use too many tokens while performing.

Thanks for the awesome work by the way!

hi，what is the reason for this error

I tried to run the demo on my computer following the instructions，anyone can help me please help me thank you very much

Not connecting to chat-gpt when running chatgpt_airsim.py

I have been attempting to try the drone simulation but when I run the chatgpt_airsim.py it gives error. I tried reinitializing everything and performed all basic debugging still no success.

What are the prerequisites for moving the tello drone through chat gpt?

I installed Anaconda after reading the previous posts, but I would appreciate it if you could tell me more about the connection procedure between the tello drone and computer or the connection procedure with chatgpt or Airsim.
I wonder if tello can be moved properly only after creating the environment related to the ChatGPT-Airsim Interface you uploaded.

https://github.com/microsoft/PromptCraft-Robotics/blob/main/chatgpt_airsim/README.md

I don't have much knowledge about coding or programming, so I'm not sure how to use the prompts you posted earlier because I don't know how.

https://github.com/microsoft/PromptCraft-Robotics/blob/main/examples/aerial_robotics/tello_example.md

I wonder if this applies to other dji drone models other than tello.

Stuck on "Initializing Airsim"

After running the simulation environment and chatgpt_airsim.py the terminal is stuck on "Initializing AirSim...". It used to work normally until a couple of days ago. Any ideas what could be the issue?

chat-robot

Broken Link

Broken link:

ChatGPT - Manipulation tasks

In Readme File at some places coordinates are wrong

The code you have provided in the readme file at certain positions of the drone while climbing the height of 50 meters to the turbine in that particular segment you have just mentioned about the co ordinate you have not moved the drone upwards . Please resolve the issue .

How to build my own simulation model through AirSim, compared to a car with RGBD camera

I have tried https://github.com/microsoft/PromptCraft-Robotics/blob/main/chatgpt_airsim/README.md，it's wonderful，and I want to build a new environment，and use chatgpt to control a car with RGBD camera to do something，how to make it come true

still on the problem and no question response？

Initializing ChatGPT...
Done.
Initializing AirSim...
what should I DO?
thank you ！

Is source code of environments open?

I want to run the experiment in linux, but these is only win version environments.

So where could we get the code of environments? it's open or in UE marketplace?

thanks