openai/neural-mmo

Code for the paper "Neural MMO: A Massively Multiagent Game Environment for Training and Evaluating Intelligent Agents"

Status: Archive (code is provided as-is, no updates expected)

Neural MMO: A Massively Multiagent Game Environment for Training and Evaluating Intelligent Agents

(9/19) This is an archive of the inital release of the Neural MMO project. Active development continuing at jsuarez5341/neural-mmo. At the time of archiving, the project is on version 1.2 and has made significant progress since v1.0 -- here is a screenshot:

Neural MMO v1.0 Release

This environment is the first neural MMO; it attempts to create agents that scale to real world complexity. Simulating evolution on Earth is computationally infeasible, but we can construct a reasonable and efficient facsimile. We consider MMORPGs (Massive Multiplayer Online Role Playing Games) the best proxy for the real world among human games: they are complete macrocosms featuring thousands of agents per persistent world, diverse skilling systems, global economies, and ad-hoc high stakes single and team based conflict.

Quickstart

# Recommended Setup:
mkdir projekt
cd projekt

git clone https://github.com/jsuarez5341/neural-mmo-client
cd neural-mmo-client
bash setup.sh
cd ..

git clone https://github.com/openai/neural-mmo
cd neural-mmo
bash scripts/setup/setup.sh
python setup.py

This will download both the OpenAI environment and the independent client, which is required for rendering. Some setups may require you fix the symlink to the client (e.g. ln -s ../../neural-mmo-client/ embyr from forge/embyr). The setup file will only install external dependencies -- we assume you have already setup Anaconda with Python 3.6+. The environment is framework independently, but our experiment code does depend on PyTorch -- set this up separately.

Run the following, then navigate to http://localhost:8080/forge/embyr/ in Firefox or Chrome to pull up the renderer. Click to start and give it a few seconds to load assets and stabilize fps. For troubleshooting, see the Client Repo.

python Forge.py --render #Run the environment with rendering on

Overview

The project is divided into four modules:

Engineering	Research
Blade: Env	Trinity: API
Embyr: Render	Ethyr: Neural

The objective is to create agents that scale to the complexity and robustness of the real world. This is a variant phrasing of "artificial life." A key perspective of the project is decoupling this statement into subproblems that are concrete, feasible, and directly composable to solve the whole problem. We split the objective into "agents that scale to their environment" and "environments that scale to the real world." These are large respective research and engineering problems, but unlike the original objective, they are specific enough to attempt individually. For a more thorough overview of the project approach and objective, see this Two Pager.

Research: Agents that scale to env complexity

Engineering: Env that scales to real world complexity

Trinity

Trinity is the native API for researchers (the naming is simply flavor -- see "Namesake" below). It consists of three base classes, Pantheon, God, and Sword, which you can override to execute code at the Cluster, Server, and Agent levels, respectively.

from forge.trinity import smith, Trinity, Pantheon, God, Sword
trinity = Trinity(Pantheon, God, Sword)
envs = smith.Native(config, args, trinity)
envs.run()

That's it -- all communications are handled internally. The provided Pantheon class includes sample gradient aggregation, optimization, and model saving code. The provided God class is mainly a stub to enable future work incorporating population level training algorithms. The provided Sword class contains our simple fully connected model. The full API is defined in forge/trinity/trinity.py.

The VecEnv/Gym computation model not well suited to this setting and performs 10-1000x more interprocess communication than is required. The native API is simpler, more efficient, and requires much less code. However, we do provide a canonical Gym API minimally modified to support a variable number of agents. Forge.py contains a complete forward pass example for a single population. Basic usage is:

from forge.trinity import smith
envs = smith.VecEnv(config, args, self.step)

#The environment is persistent: call reset only upon initialization
obs = envs.reset()

#Observations contain entity and stimulus
#for each agent in each environment.
actions = your_algorithm_here(obs)

#The environment is persistent: "dones" is always None
#If an observation is missing, that agent has died
obs, rewards, dones, infos = envs.step(actions)

You can try these both out with:

# Run Options:
python Forge.py --nRealm 2 --api native #Run 2 environments with native API
python Forge.py --nRealm 2 --api vecenv #Run 2 environments with vecenv API

Ethyr

Ethyr is the "contrib" for this project. It contains useful research tools for interacting with the project. I've seeded it with the helper classes for our experiments, including a model save/load manager, a rollout objects, and a basic optimizer. If you would like to contribute code (in any framework, not just PyTorch), please submit a pull request.

Blade

Blade is the core environment, including game state and control flow. Researchers should not need to touch this, outside perhaps importing core configurations and enums.

Embyr

Embyr is the independent THREE.js web client. This is downloaded from a separate repository in the setup above and symlinked to the OpenAI repository. You may have to fix the symbolic link if you get forge/embyr import errors. Again, researchers should not have to edit this. In order to run it, run Forge.py with --render enabled, then navigate to localhost:8080/forge/embyr in Firefox. It will take a couple seconds to initialize and load assets. You will need to refresh the page whenever you reboot the server (Forge.py). Chrome and Safari might work, but we do not currently offer official support.

Failure Modes

Evaluation can be somewhat difficult in our setting but is not a major blocker. For smaller experiments, we find population size and resource utilization to be reasonable metrics of success. For larger experiments with sufficient domain randomization, Tournaments (as described in the accompanying paper) allow for cross validation of approaches.

We are currently aware of three failure cases for the project:

• Computational infeasibility
• "Agents that scale to their environment" is too hard
• "Environments that scale to the real world" is too hard

The first failure case is a serious risk, but is shared among all areas of the field. This project is not uniquely compute intensive -- in fact, it is one of few environments where it is straightforward to train reasonable policies on a single CPU. If scale is the main issue here, it is likely shared among most if not all other approaches.

The second problem is probably most familiar to researchers as exploration. Given a cold start, how can agents bootstrap both to better policies and to better exploration strategies? This is a hard problem, but it is unlikely to kill the project because:

• This is independently an important problem that many researchers are already working on already
• The environment of this project is designed collaboratively to assist agents early on in learning, rather than adversarially as a hard benchmark
• Recent projects have demonstrated success at scale.

The third problem probably appears most likely to many researchers, but least likely to anyone who has spent a significant amount of time in MMOs. Here is a map of the NYC subway:

Source

Actually, it's a quest map of Runescape, a particular MMO that our environment is loosely based upon. Each quest is a puzzle in itself, takes anywhere from several minutes to several hours to complete, is part of an interconnected web of prerequisites of other quests, and provides different incentives for completion ranging from equipment to unlockable content to experience in a tightly connected set of skills:

Source

In a massive open world:

Source

The most complex class of games considered to date is MOBAs (Massive Online Battle Arenas, e.g. Dota, Quake CTF), which are round based, take on order of an hour, and are mechanically intensive. Achieving 99 in all skills and acquiring the best gear in Runescape takes, at minimum, several thousand hours. In a tournament setting where attacking other players is allowed everywhere, moment-to-moment gameplay is less important than balancing the risks and rewards of any potential strategy--especially in the presence of hundreds of other players attempting to do the same. There is almost certainly still a complexity gap from MMOs to the real world, but we believe it is much smaller than that in environments currently available.

While our environment is nowhere near the level of complexity of a real MMO yet, it does contain key properties of persistence, population scale, and open-endedness. As agents begin to reach the ceiling of the current environment, we plan on continuing development to raise the ceiling.

File Structure

/forge/trinity ~350 lines

• /forge/trinity/ann.py - Defines architectures
• /forge/trinity/god.py - Defines server level code (e.g. entity tagging)
• /forge/trinity/pantheon.py - Defines cluster level code (e.g. gradient averaging)
• /forge/trinity/sword.py - Defines core level code (e.g. running networks, collecting rollouts, computing gradients)
• /forge/trinity/trinity.py - Wraps a pantheon, god, and sword
• /forge/trinity/smith.py - Defines the Native and VecEnv / Gym APIs

/forge/ethyr ~250 lines

• /forge/ethyr/rollouts.py - Collects and merges rollouts
• /forge/ethyr/stim.py - Produces a stimulus from local game state
• /forge/ethyr/torch - pytorch specific neural utilities
  • /forge/ethyr/torch/loss.py - Defines policy/value loss and advantage
  • /forge/ethyr/torch/optim.py - Defines optimization and gradient computation
  • /forge/ethyr/torch/param.py - Network surgery useful for serialization
  • /forge/ethyr/torch/stim.py - Wraps the generic stimulus library with pytorch tensors
  • /forge/ethyr/torch/utils.py - Generic pytorch tools

/forge/blade ~2k lines, of which >1k are for future expansion. Only italicized files are relevant.

• /forge/blade/action - Defines what entities can do, what occurs when they do it, and provides structure for reasoning over actions.
  • /forge/blade/action/action.py - Class stubs for each action
  • /forge/blade/action/tree.py - Tree structure for assembling actions (e.g. action -> sub-action -> args)
  • /forge/blade/action/v2.py - Actions that entities can select, instantiate, and .call() to directly modify world state
• /forge/blade/core — Contains ~500 lines of state and game loop code.
  • /forge/blade/core/config.py - Defines specifications for each test environment, including entity statistics, the map, and spawn locations.
  • /forge/blade/core/tile.py - Defines an individual game tile
  • /forge/blade/core/map.py - Defines a map of game tiles
  • /forge/blade/core/env.py - Wraps the full game state
  • /forge/blade/core/realm.py - Defines the game loop updating the environment and agents.
• /forge/blade/entity - Defines agents, both neural and scripted
  • /forge/blade/entity/player.py — Defines "player" state
  • /forge/blade/entity/npc/
    • /forge/blade/entity/npc/npc.py — This defines client state (e.g. wraps the neural net making decisions)
    • /forge/blade/entity/npc/mobs.py - Defines scripted npcs
• /forge/blade/hook - Defines startup scripts that preassemble references
• /forge/blade/item - Defines all items in the game, including equipment
• /forge/blade/lib - Defines generic utilities that either do not have clean python implementations or require specific changes for the project
• /forge/blade/systems - Defines game content

/forge/embyr See Client Repo

Namesake

In formal publications, we refer to our project as simply a "Neural MMO." Internally and informally, we call it "Projekt: Godsword." The name comes from two sources: CD Projekt Red, my personal favorite game dev studio, and OldSchool Runescape, which contains an iconic set of weapons called god swords. The latter is a particularly good model for AI environments; the former is more of a soft flavor inspiration.

Authorship

I, Joseph Suarez, am the author of the core code base. Yilun Du assisted with running experiments and particularly in setting up tournaments. Phillip Isola and Igor Mordatch have been invaluable collaborators and advisers throughout the project. The client was developed independently as a collaboration between myself and Clare Zhu.

License & Assets

The OpenAI repository is open sourced under the MIT License. There is a smaller original code base and game kernel that I (Joseph Suarez) retain ownership of, along with associated ideas. I created these before my employment -- the initial commit here represents the latest pre-employment timestep.

Some assets used in this project belong to Jagex , the creators of Runescape, such as

We currently use them for flavor as an homage to the game that inspired the project. We believe these fall under fair use as a not-for-profit project for the advancement of artificial intelligence research -- however, we are more than happy to remove them upon request. We do own the 2D and 3D files for agents.

neural-mmo's Issues

Help : No module named 'ray'

(tensorflow) D:\git\projekt\neural-mmo>python setup.py
Traceback (most recent call last):
File "setup.py", line 6, in
import terrain
File "D:\git\projekt\neural-mmo\terrain.py", line 5, in
from forge.blade.lib import enums
File "D:\git\projekt\neural-mmo\forge\blade\lib_init_.py", line 4, in
from .ray import init
File "D:\git\projekt\neural-mmo\forge\blade\lib\ray.py", line 1, in
import ray, os
ModuleNotFoundError: No module named 'ray'

pip install $pkg --user

Since (at least for some people) pip requires --user and while I'm sure there is a better way to do this:

cat neural-mmo/scripts/setup/setup.sh | grep "pip" | sed 's/$/ --user/' | bash

Where would be the best place to put this information? I know it's trivial but it's a lot better than editing line by line. Could we have a wiki with tips or something of that sort?

NOTICE: Please raise issues at jsuarez5341/neural-mmo

This repo contains an achieved version of Neural MMO. The project is still under active development with full support and major updates every few months -- see my fork at jsuarez5341/neural-mmo for the latest official version. I no longer have permissions to merge changes submitted to the OpenAI repo.

Could I visualize the training process without --render?

Hi, thanks for opensource this excellent work! I have tried the example and it works very well with --render. When I digged into the code, with opening render option, it seems the client code was invoked, and I had trouble to find the training process in the render code. Without render, I got the path of training procedure. but running without render always stuck in run function of NtiveRealm because the update would always be None, and program fall into the endless loop. I am trying to figure out if there is a API that I can observe the training process without --render option on? Thank you very much!

ImportError: cannot import name 'factorial' from 'scipy.misc'

i'm just trying to run this:
import statsmodels.formula.api as sm

then i get this error:
import statsmodels.formula.api as sm
Traceback (most recent call last):

File "", line 1, in
import statsmodels.formula.api as sm

File "c:\users\ramma\appdata\local\programs\python\python37-32\lib\site-packages\statsmodels\formula\api.py", line 15, in
from statsmodels.discrete.discrete_model import MNLogit

File "c:\users\ramma\appdata\local\programs\python\python37-32\lib\site-packages\statsmodels\discrete\discrete_model.py", line 45, in
from statsmodels.distributions import genpoisson_p

File "c:\users\ramma\appdata\local\programs\python\python37-32\lib\site-packages\statsmodels\distributions_init_.py", line 2, in
from .edgeworth import ExpandedNormal

File "c:\users\ramma\appdata\local\programs\python\python37-32\lib\site-packages\statsmodels\distributions\edgeworth.py", line 7, in
from scipy.misc import factorial

ImportError: cannot import name 'factorial' from 'scipy.misc' (c:\users\ramma\appdata\local\programs\python\python37-32\lib\site-packages\scipy\misc_init_.py)

Renderer Problem

When I pull up the renderer, the LegacyWebClient cannot show the real-time picture but only the waiting screen

The code report errors as

client only shows background (no map) when using Docker

Hi all - great work on this package. I'm able to setup the environment perfectly on a local laptop - everything works! Also, love the flavor! RS was a lot of fun:

I'm trying to set this up on a remote server using Docker and everything works until the Client Render. The "title screen" will pop up and upon clicking will show the green background - however the map won't render. I see the following image:

Any thoughts? The below are steps for reproducing the work and setup.

Sample Dockerfile:

FROM nvcr.io/nvidia/pytorch:19.02-py3

RUN mkdir /workspace/explore-neural-mmo
ADD . /workspace/explore-neural-mmo

RUN git clone https://github.com/jsuarez5341/neural-mmo-client.git /workspace/explore-neural-mmo/neural-mmo-client  &&\
    cd /workspace/explore-neural-mmo/neural-mmo-client &&\
    bash setup.sh

RUN git clone https://github.com/openai/neural-mmo.git /workspace/explore-neural-mmo/neural-mmo &&\
    cd /workspace/explore-neural-mmo/neural-mmo &&\
    bash scripts/setup/setup.sh &&\
    python setup.py

RUN pip install jupyterlab pygame autobahn

WORKDIR /workspace/explore-neural-mmo

CMD ["bash"]

Build and run:

docker build -t neural_mmo:latest .

nvidia-docker run -it --rm \
	  -p 6006:6006 -p 8080:8080 -p 8888:8888 \
	  neural_mmo:latest

Run a simple environment:

python Forge.py --render

How the Ubuntu system to use client?

It is success that ues the client in the windows， but I don't konw how to use it in the Ubuntu system?

No module named 'opensimplex'

When trying to install, at the very last line 'python setup.py', getting the above error. I'm on Win10, have conda installed (python 3.7). Didn't have any problems installing in the neural-mmo-client folder. For the neural-mmo folder, I used the Git command 'sh scripts/setup/setup.py' for the bash install. Any ideas where it went wrong?

Does not work on Windows

It seems that neural-mmo is not available at Windows, due to the difficulties with the Ray library. Is there any way to solve it?

Not able to run the experiments.

How do I run the Forge for a different amount of participants. I tried changing values in experiments.py but it just makes empty folders in the resources/exps.

This is the error I run into when I run Forge.py
FileNotFoundError: [Errno 2] No such file or directory: 'resource/exps/testchaos128/model/bests.pth'

No module named 'forge.embyr'

Hi, it is a really awsome project. I want to setup it on my server.
And following the instructions I have installed all dependencies.
But when I run

python3 Forge.py --render

It occurs errors.

Traceback (most recent call last):
  File "Forge.py", line 72, in <module>
    example.env.render()
  File "/root/mmo/neural-mmo/forge/trinity/smith.py", line 57, in render
    from forge.embyr.twistedserver import Application
ModuleNotFoundError: No module named 'forge.embyr'

Page stuck

After I run
python Forge.py
Terminal shows out:
python Forge.py --render testlaw128 , NENT: 128 , NPOP: 8 testchaos128 , NENT: 128 , NPOP: 8 sample , NENT: 128 , NPOP: 8 Enabling local test mode for render Initializing new model... Loading model... #Params: 1484.264 K pygame 1.9.4 Hello from the pygame community. https://www.pygame.org/contribute.html Uptime: 0.6 , Tick: 1 Uptime: 1.2 , Tick: 2 Uptime: 1.8 , Tick: 3 Uptime: 2.4 , Tick: 4 Uptime: 3.0 , Tick: 5 Uptime: 3.6 , Tick: 6 Uptime: 4.2 , Tick: 7 Uptime: 4.8 , Tick: 8 Created a server WebSocket connection request: {"peer": "tcp4:127.0.0.1:37968", "headers": {"host": "localhost:8080", "user-agent": "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:66.0) Gecko/20100101 Firefox/66.0", "accept": "*/*", "accept-language": "en-US,en;q=0.5", "accept-encoding": "gzip, deflate", "sec-websocket-version": "13", "origin": "http://localhost:8080", "sec-websocket-extensions": "permessage-deflate", "sec-websocket-key": "IZS4kvPcZ2OdzO76g41ohg==", "connection": "keep-alive, Upgrade", "pragma": "no-cache", "cache-control": "no-cache", "upgrade": "websocket"}, "host": "localhost", "path": "/ws", "params": {}, "version": 13, "origin": "http://localhost:8080", "protocols": [], "extensions": [["permessage-deflate", {}]]} Opened connection to server Uptime: 5.4 , Tick: 9 Uptime: 6.0 , Tick: 10 Uptime: 6.6 , Tick: 11 Uptime: 7.2 , Tick: 12 Uptime: 7.8 , Tick: 13 Uptime: 8.4 , Tick: 14 Uptime: 9.0 , Tick: 15 Uptime: 9.6 , Tick: 16 Uptime: 10.2 , Tick: 17 Uptime: 10.8 , Tick: 18
But the page of http://localhost:8080/forge/embyr/ are shown as

And no reaction whatever I click.

Issue on file tile.py unreachable return statement

Error is at lines 45 through 47 not sure what should be indented.

def harvest(self):
      if self.capacity == 0:
         return False
      elif self.capacity <= 1:
         self.state = self.mat.degen()
      self.capacity -= 1
      return True
      return self.mat.dropTable.roll()

Integration with MADDPG

I'm trying to integrate Neural MMO with MADDPG (https://github.com/openai/maddpg ; https://github.com/openai/multiagent-particle-envs ) and am facing some issues related to this:

• Native Route

  • When running Neural MMO from the command line, this does not appear to work when the "-- render" argument is omitted (i.e., just "python Forge.py" is called). It stops after the "Hello from the pygame community" line.
  • Is there even a way to do this using Native? I understand for MADDPG, a situation like VecEnv is needed where actions, observations, reward, and world are all compartmentalized. If I decompartmentalize some of them, such as separating out the super() call from NativeRealm from the non-super part, it may work. It appears in general the paradigm is different here, however, I have made some other modifications and they work in Native, so this would be great if it could be used for MADDPG integration.

• VecEnv/Gym Route

  • When running Neural MMO with vecenv ("python Forge.py --api vecenv --render") and launching the browser graphical display, no agents show up. I understand this works slower than Native, so is it the case that if I wait long enough / get a more powerful machine, then it will work?
  • When running "python Forge.py --api vecenv", it hangs at the "Hello from the pygame community" part just like the Native version and then gives an error. I'll include the error at the end of this message.
  • This seems like it could interface with MADDPG more easily, however, because of the issues above, I can't do much with it.

• In general, what is the best practice for integrating with MADDPG in your view? Has anyone integrated Neural MMO with, say, PPO or DPG or something similar? I am somewhat new to this field, so apologies for any ignorance and thanks in advance for your patience!

Error encountered when running vecenv with no rendering

testlaw128 , NENT: 128 , NPOP: 8
testchaos128 , NENT: 128 , NPOP: 8
sample , NENT: 128 , NPOP: 8
2019-05-10 18:03:39,816 INFO node.py:423 -- Process STDOUT and STDERR is being redirected to /tmp/ray/session_2019-05-10_18-03-39_2786/logs.
2019-05-10 18:03:40,032 INFO services.py:363 -- Waiting for redis server at 127.0.0.1:64431 to respond...
2019-05-10 18:03:40,257 INFO services.py:363 -- Waiting for redis server at 127.0.0.1:28843 to respond...
2019-05-10 18:03:40,274 INFO services.py:760 -- Starting Redis shard with 6.86 GB max memory.
2019-05-10 18:03:40,455 INFO services.py:1384 -- Starting the Plasma object store with 10.29 GB memory using /dev/shm.
(pid=2818) pygame 1.9.5
(pid=2818) Hello from the pygame community. https://www.pygame.org/contribute.html
Traceback (most recent call last):
File "Forge.py", line 74, in
example.run()
File "Forge.py", line 48, in run
self.step()
File "Forge.py", line 44, in step
self.envsObs, rews, dones, infos = self.env.step(actions)
File "/home/kumar/Projects/neural-mmo/forge/trinity/smith.py", line 70, in step
return self.env.step(actions)
File "/home/kumar/Projects/neural-mmo/forge/trinity/smith.py", line 23, in step
zip(self.envs, actions)])
File "/home/kumar/.local/lib/python3.6/site-packages/ray/worker.py", line 2307, in get
raise value
ray.exceptions.RayTaskError: ray_VecEnvRealm:step() (pid=2818, host=dvrs-10)
File "/home/kumar/Projects/neural-mmo/forge/blade/core/realm.py", line 176, in step
return pickle.dumps((stims, rews, None, None))
File "/home/kumar/Projects/neural-mmo/forge/blade/entity/player.py", line 68, in getattribute
return super().getattribute(name)
RecursionError: maximum recursion depth exceeded while calling a Python object

Missing map.tmx

It seems that running Forge.py requires the file "map.tmx", which is missing
I am wondering if this is a problem of my configuration or other things

No such file or directory: 'resource/maps/procedural/map0/map.tmx'

Logs don't get updated

The logs folder doesn't update. Moreover how to generate models for 16, 35, 64, etc agents. I am only able to run it for 128 number of agents as I don't have example models for the rest. @jsuarez5341 please help.

ImportError: cannot import name 'imread' from 'scipy.misc'

When running this step:
]$ python setup.py
Traceback (most recent call last):
File "setup.py", line 6, in
import terrain
File "/home/rsigler/src/neural-mmo/neural-mmo/terrain.py", line 7, in
from scipy.misc import imread, imsave
ImportError: cannot import name 'imread' from 'scipy.misc' (/usr/local/anaconda2/envs/python3/lib/python3.7/site-packages/scipy/misc/init.py)

]$ ipython
Python 3.7.2 (default, Dec 29 2018, 06:19:36)
Type 'copyright', 'credits' or 'license' for more information
IPython 7.2.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import scipy.misc

In [2]: dir(scipy.misc)
Out[2]:
['all',
'builtins',
'cached',
'doc',
'file',
'loader',
'name',
'package',
'path',
'spec',
'_comb',
'_fact',
'_fact2',
'_factk',
'_info',
'_lsm',
'_msg',
'_pade',
'_source',
'_who',
'absolute_import',
'ascent',
'central_diff_weights',
'comb',
'derivative',
'division',
'doccer',
'electrocardiogram',
'face',
'factorial',
'factorial2',
'factorialk',
'info',
'logsumexp',
'np',
'pade',
'print_function',
'source',
'test',
'who']

In [3]:

Agent pass the stone

Hi, when i try to render the env, I find that agent pass the stone.

Might be a bug in ethyr/stim.py

As i gathered, stim.py is about collecting features for an agent to make a decision in a particular state. My question is about tile function. This is called in environment function for each tile, visible for ent. If this tile is not empty, i.e. has entities on it, stats of this entity are appended to a list. Two issues with how it is done:

1. This might be a bug. Right now these stats of entities are collected with statStim = stats(ent, e, config). However, ent is the agent that makes the decision, and e is either itself or its neighbour. So, if we look at stats, right now this statStim = stats(ent, e, config) collects the same stats of ent over and over again and only the distance from ent to e is collected correctly. Shouldn't this be statStim = stats(e, ent, config) instead, so stats of e are collected? Like, hp, food, and water of neighbours, instead of the agent itself repeatedly?
2. This is more of a suggestion. Currently, information about ent is always included in the list of entities. May be it's better to check if e.entID != ent.entID and not add ent to the list, unless as a placeholder when there are no neighbours?

Here's the code with two modifications included (I did not run it, but it at least illustrates the concept)

def environment(env, ent, sz, config):
R, C = env.shape
conv, ents = np.zeros((2, R, C)), []
for r in range(R):
for c in range(C):
t, e = tile(ent, env[r, c], sz, config)
conv[:, r, c] = t
ents += e
if len(ents) == 0: # this is also new
ents.append(stats(ent, ent, config))
assert len(ents) > 0
ents = np.array(ents)
return conv, ents

def tile(ent, t, sz, config):
nTiles = 8
index = t.state.index
assert 0 <= index < nTiles
conv = [index, t.nEnts]

ents = []
r, c = ent.pos
for e in t.ents.values():

    # my modification instead (continued in environment):
    if ent.entID != e.entID:
        statStim = stats(e, ent, config)  # this is the most important suggestion
        e.stim = statStim
        ents.append(statStim)

return conv, ents

Agents don't attack each other

when I run with NPOP=2(or more), NENT=128 and --render, I never see the agents attacking each other during my whole observation. I'm still new to this project, and the only attempt I've made is to change the root in forge.ethyr.torch.io.action.leaves from action.Move to action.Action or action.Attack, but it ends up with errors like "list index out of range". Should I modify the action and the network myself, or is that any simple way I can make them combat like in the demo?

The configure of multi-cores

Hello,

This is not a mistake, but I feel confused about the experiments.py, the document claims that there is a setting for the multi-core.

However, I can not figure out what's the parameter here.

is the
#Hardware specification
DEVICE = 'cpu:0'
part?

If yes, what should I do if I want to set more CPUs.

BTW, is the nerual-mmo support the GPU？

Where is API?

I would like to use C# to play neural mmo, but cant find API to interact with game server. Where is it?

mmo architecture

Thanks for developing.
After reviewing the code, I have these questions and concerns:

1. Is there any document that shows the detailed architecture of mmo?
2. MMO and MMO-client can run on different machine, i.e., one is server, the other is user interface that shows the rendering/training result. Also, it is centralized, which means in MMO all agents face the same "environment". Am I correct?
3. Is there any limit of maximum number of agent that MMO can run? If the limit exists, how it is determined? Are they determined based on game rules?
4. Does this repo accept community contributions?

https://github.com/openai/neural-mmo first

The first link on this page is no longer a valid link that days recent projects have been a success.

GPU acceleration on training and rendering

Based on my test run on Forge. The training and rendering speed are slow (I run it on a Macbook pro). As MMO is using Pytorch framework, is it possible to support GPU acceleration to boost training as well as rendering.

how to change the 3d version for the render

i can only see the render in a plain，how to control the version?

neural-mmo does not work on Windows

#2 pointed out difficulties with the Ray library. As far as I know, it is not available at all for Windows, cf. https://news.ycombinator.com/item?id=16512784

Plan A would be to make neural-mmo run normally on Windows. This is probably more trouble than it is worth (porting Ray or finding an alternative).

Plan B would be to salvage what can be, ensuring that setup.py runs even without Ray. Of course, this option makes sense only if, say, some rendering is missing, not the core model (as I could not finish said setup, I don't know what would still be working).

Plan C would be informing Windows users that neural-mmo is not going to work (so that they save the trouble to try).

Frames Per Second Per Core?

(Note: This is a question, not a bug or issue with function/performance! Also, wasn't sure where else to ask this, so apologies if this is the wrong place - kindly let me know the proper place if there is one!)

I found that Facebook ELF (https://github.com/facebookresearch/ELF) lists the number of (AI) frames per second per core (40k). What is the frames per second per core for Neural MMO? When running this code, how many cores are used? Thank you.

Thank you to everyone who worked on this. This is quite amazing and you should all be proud of the great work you have done!

Forge.py error

when I tried to run python Forge.py --render, I got this:

Traceback (most recent call last):
  File "Forge.py", line 5, in <module>
    import experiments
  File "/Users/andychen/projekt/neural-mmo/experiments.py", line 2, in <module>
    from configs import Law, Chaos
  File "/Users/andychen/projekt/neural-mmo/configs.py", line 2, in <module>
    from forge.blade.core.config import Config
  File "/Users/andychen/projekt/neural-mmo/forge/blade/core/__init__.py", line 1, in <module>
    from .env import Env
  File "/Users/andychen/projekt/neural-mmo/forge/blade/core/env.py", line 8, in <module>
    from forge.blade import systems
  File "/Users/andychen/projekt/neural-mmo/forge/blade/systems/__init__.py", line 3, in <module>
    from .recipe import Recipe
  File "/Users/andychen/projekt/neural-mmo/forge/blade/systems/recipe.py", line 4
    def __init__(self, *args, amtMade=1):
                                    ^
SyntaxError: invalid syntax