YouTuber Spends Months Building a 400 Million Block ChatBot in Minecraft That Takes Hours to Respond
- CraftGPT is a giant redstone chatbot in Minecraft with 5 million parameters, but has limited vocabulary.
- Includes over 430 million blocks, runs on a special high-speed server.
- Vanilla Minecraft updates with Copper Age, adds copper gear, golems, and community challenges.
Jump to:
Blocks as Far as The Eye Can See
Minecraft continues to prove itself fertile ground for creativity, as a YouTuber known for ambitious builds, sammyuri, has revealed “CraftGPT.” This is an AI Small Language Model (SML) built entirely within Minecraft, that’s capable of producing simple answers to questions…although it can take two hours to get a response.
Sammyuri revealed CraftGPT in a recent video, where he noted that it “was built over a period of many months” and takes up an absolutely massive in-game size of 1,020x260x1,656 blocks, so the Distant Horizons mod (which increases view distance without harming performance) was used to capture full footage of its scale.
“The model has 5,087,280 parameters, trained in Python on the TinyChat dataset of basic English conversations,” sammyuri said. “It has an embedding dimension of 240, vocabulary of 1920 tokens, and consists of 6 layers. The context window size is 64 tokens, which is enough for (very) short conversations.”
In English, Please
In short, sammyuri used Python within Minecraft to help build CraftGPT with code. By doing so, he could use a preexisting language-training dataset so that his SML would understand basic instructions and conversations. “Tokens” in this context refers to text, which can be one or multiple words; CraftGPT has a very, very small selection of vocabulary compared to GPT-style Large Language Models (LLM), which typically use between 30,000 – 50,000 tokens.
“Layers” effectively refers to the neural building blocks of the SML; this has six, whereas ChatGPT uses 96. The higher the number, the more refined an answer can be within a language model.
Sammyuri also highlighted how his model uses “weights (learned values) quantized to 8 bits, although the embedding and LayerNorm weights are stored at 18 and 24 bits respectively.” Bits represent the numerical values, or parameters, that determine the model’s behavior, with higher bits (16 and 32 for GPT) effectively meaning the language model is more precise.
Redstone Builds
Gamers not familiar with the programming abilities built into Minecraft may at least be familiar with redstone, the block that allows players to do pretty much anything from crafting pressure-pad activated doors through to, well, this SML. Sammyuri is one of the most well-known redstone builders in the community, with previous projects including Minecraft in Minecraft and a 1Hz CPU inside Minecraft that could run games like Snake.
He’s worked with other noted creators, including Uwerta and StackDoubleFlow, and in all his videos has explained that, for the things he builds to actually run, he uses the special Minecraft High Performance Redstone Server (MHPRS). This can speed-up redstone processing considerably, with CraftGPT running at 40,000x speed, for example.
Copper Blocks
MHPRS lacks a lot of vanilla Minecraft features, so is best for gamers who like to experiment with programming-related builds. However, normal Minecraft players can still experiment with building new things as developers Mojang recently released the third big update this year: The Copper Age. As its name suggests, this update gives players a reason to mine copper beyond turning it into decorative items. It can now become weapons, tools, armor and even a chunky little golem. The copper golem also oxidizes over time, eventually turning into a statue that can be used with a redstone comparator.
Minecraft is running special copper-themed community challenges at the moment, too. The first one is live until Oct. 6, and requires everyone to make as many copper golems as possible – once 75 million have been made (the total number, not by each person) it’ll unlock a Copper Golem Plushie Character Creator item in Bedrock (the multiplatform version) and the Copper Chemist skin in Java (the original PC version).
Mojang also ran the latest Minecraft Live event on YouTube recently, with news of another update, Mounts of Mayhem, revealing that the spear is being introduced. This will be the first new tiered weapon since the hoe appeared in 2010. Spears can be used for jousting, but not just by players. Players can expect underwater spear fights soon, as enemies will also get their own zombie version of the new mount, the aquatic nautilus.
or
