-
Notifications
You must be signed in to change notification settings - Fork 7.7k
Home
ThiloteE edited this page Aug 28, 2024
·
15 revisions
GPT4All - Your private local LLM environment, brought to you by NOMIC.
Welcome to the GPT4All Wiki! We're excited to bring you an open-source project that allows you to run large language models (LLMs) privately on your own computer. With GPT4All, you can chat with LLMs and integrate them into your workflow without relying on cloud services.
GPT4All is designed for Windows, macOS, and Linux users.
- CPU: GPT4All installers require your CPU has AVX/AVX2 instruction sets.
- Resolution: You need a display resolution of at least 1280x720.
- Memory: At least 8 GB of system RAM.
-
OS: A recent Operating System...
- Windows 10 or later
- macOS High Sierra (10.13) or later
- Ubuntu 22.04 LTS or later
Have enough of it, because ...
- the large language model (LLM) should fit into RAM completely. Reason being: Trying to load a model that does not fit into your RAM triggers your machine to utilize the swap space (assuming there is one) on your harddrive (SSD/HDD) and that will slow down speed of inference substantially. In short: RAM is faster than your harddrive (HDD/SSD).
- chatting with the model adds to the context, which is mapped into RAM. The longer the conversation, the more RAM is required.
- more RAM will allow you to run larger models with larger context.
Have one with lots of VRAM, because ...
- GPU are very fast at inferenceing LLMs and in most cases faster than a regular CPU / RAM combo.
- We recommend at least 8GB of VRAM.
Have one that is supported by the GPU backends:
- Nvidia
- CUDA backend
- will run any .gguf quantized models.
- available for the LocalDocs feature
- Vulkan Backend
- will run .gguf quantized models of fp16, Q4_0, Q4_1.
- CUDA backend
- AMD
- Vulkan Backend
- will run .gguf quantized models of fp16, Q4_0, Q4_1.
- Vulkan Backend
| CPU (AVX/AVX2) |
CPU (ARM NEON) |
Metal (Apple) |
Vulkan/Kompute (AMD/Nvidia) |
Cuda (Nvidia) |
|
|---|---|---|---|---|---|
| GGUF Q4_0, Q4_1 & F16 |
✅ | 🚫 | ✅ | ✅ | ✅ |
| GGUF K-quants |
✅ | 🚫 | ✅ | 🚫 | ✅ |
| GGUF I-quants |
✅ 🐢 | 🚫 | ✅ | 🚫 | ✅🐢 |
| GGUF K cache quants |
❓ | 🚫 | ❓ | ❓ | ❓ |
| Multi-GPU | N/A | N/A | ❓ | ❓ | ❓ |
- ✅: feature works
- 🚫: feature does not work
- ❓: unknown, please contribute if you can test it youself
- 🐢: feature is slow
- Download the GPT4All installer for Windows
- Download the GPT4All installer for macOS
- Download the GPT4All installer for Ubuntu
We're here to help!
- Check out the troubleshooting information here.
- See our website documentation.
- Report issues and bugs at GPT4All GitHub Issues.
- Join the GitHub Discussions
- Ask questions in our discord chanels