Crafting minds for Minecraft with LLMs and Mineflayer!

FAQ | Discord Support | Video Tutorial | Blog Post | Contributor TODO

• Minecraft Java Edition (up to v1.21.1, recommend v1.20.4)
• Node.js Installed (at least v14)
• One of these: OpenAI API Key | Gemini API Key | Anthropic API Key | Replicate API Key | Hugging Face API Key | Groq API Key | Ollama Installed. | Mistral API Key | Qwen API Key [Intl.]/[cn] | Novita AI API Key |

1. Make sure you have the requirements above.

2. Clone or download this repository (big green button)

3. Rename keys.example.json to keys.json and fill in your API keys (you only need one). The desired model is set in andy.json or other profiles. For other models refer to the table below.

4. In terminal/command prompt, run npm install from the installed directory

5. Start a minecraft world and open it to LAN on localhost port 55916

6. Run node main.js from the installed directory

If you encounter issues, check the FAQ or find support on discord. We are currently not very responsive to github issues.

You can configure project details in settings.js. See file.

You can configure the agent's name, model, and prompts in their profile like andy.json with the model field. For comprehensive details, see Model Specifications.

If you use Ollama, to install the models used by default (generation and embedding), execute the following terminal command: ollama pull llama3 && ollama pull nomic-embed-text

To connect to online servers your bot will need an official Microsoft/Minecraft account. You can use your own personal one, but will need another account if you want to connect too and play with it. To connect, change these lines in settings.js:

"host": "111.222.333.444",
"port": 55920,
"auth": "microsoft",

// rest is same...

To use different accounts, Mindcraft will connect with the account that the Minecraft launcher is currently using. You can switch accounts in the launcer, then run node main.js, then switch to your main account after the bot has connected.

If you intend to allow_insecure_coding, it is a good idea to run the app in a docker container to reduce risks of running unknown code. This is strongly recommended before connecting to remote servers.

docker run -i -t --rm -v $(pwd):/app -w /app -p 3000-3003:3000-3003 node:latest node main.js

or simply

docker-compose up

When running in docker, if you want the bot to join your local minecraft server, you have to use a special host address host.docker.internal to call your localhost from inside your docker container. Put this into your settings.js:

"host": "host.docker.internal", // instead of "localhost", to join your local minecraft from inside the docker container

To connect to an unsupported minecraft version, you can try to use viaproxy

Bot profiles are json files (such as andy.json) that define:

1. Bot backend LLMs to use for talking, coding, and embedding.
2. Prompts used to influence the bot's behavior.
3. Examples help the bot perform tasks.

LLM models can be specified simply as "model": "gpt-4o". However, you can use different models for chat, coding, and embeddings. You can pass a string or an object for these fields. A model object must specify an api, and optionally a model, url, and additional params.

"model": {
  "api": "openai",
  "model": "gpt-4o",
  "url": "https://api.openai.com/v1/",
  "params": {
    "max_tokens": 1000,
    "temperature": 1
  }
},
"code_model": {
  "api": "openai",
  "model": "gpt-4",
  "url": "https://api.openai.com/v1/"
},
"embedding": {
  "api": "openai",
  "url": "https://api.openai.com/v1/",
  "model": "text-embedding-ada-002"
}

model is used for chat, code_model is used for newAction coding, and embedding is used to embed text for example selection. If code_model or embedding are not specified, they will use model by default. Not all APIs have an embedding model.

All apis have default models and urls, so those fields are optional. The params field is optional and can be used to specify additional parameters for the model. It accepts any key-value pairs supported by the api. Is not supported for embedding models.

Embedding models are used to embed and efficiently select relevant examples for conversation and coding.

Supported Embedding APIs: openai, google, replicate, huggingface, novita

If you try to use an unsupported model, then it will default to a simple word-overlap method. Expect reduced performance, recommend mixing APIs to ensure embedding support.

By default, the program will use the profiles specified in settings.js. You can specify one or more agent profiles using the --profiles argument: node main.js --profiles ./profiles/andy.json ./profiles/jill.json

Some of the node modules that we depend on have bugs in them. To add a patch, change your local node module file and run npx patch-package [package-name]

@misc{mindcraft2023,
    Author = {Kolby Nottingham and Max Robinson},
    Title = {MINDcraft: LLM Agents for cooperation, competition, and creativity in Minecraft},
    Year = {2023},
    url={https://github.com/kolbytn/mindcraft}
}