Skip to content

🧠 两小时从 0 训出你的 64M 大模型。Train your own 64M LLM from scratch in 2 hours.

update readme

gongjy5dbd6174b3ab
diff --git a/README.md b/README.md
index 4842822..73f9406 100644
--- a/README.md
+++ b/README.md
@@ -25,9 +25,9 @@
</div>
-* 本开源项目旨在完全从0开始,最快仅用3小时!即可训练出仅为26M大小的微型语言模型**MiniMind**。
-* **MiniMind**极其轻量,体积约是 GPT3 的 $\frac{1}{7000}$,力求做到最普通的个人GPU也可快速推理甚至训练。
-* **MiniMind**改进自DeepSeek-V2、Llama3结构,项目包含整个数据处理、pretrain、sft、dpo的全部阶段,包含混合专家(MoE)模型。
+* 本开源项目旨在完全从0开始,最快仅用3小时!即可训练出仅为26.88M大小的微型语言模型**MiniMind**。
+* **MiniMind**极其轻量,最小版本体积约是 GPT3 的 $\frac{1}{7000}$,力求做到最普通的个人GPU也可快速推理甚至训练。
+* **MiniMind**发布了大模型极简结构,数据集清洗和预处理、监督预训练(Pretrain)、有监督指令微调(SFT)、低秩自适应(LoRA)微调,无奖励强化学习直接偏好对齐(DPO)的全阶段代码,也包含拓展共享混合专家(MoE)的稀疏模型;拓展视觉多模态VLM: [MiniMind-V](https://github.com/jingyaogong/minimind-v)。
* 这不仅是一个开源模型的实现,也是入门大语言模型(LLM)的教程。
* 希望此项目能为研究者提供一个抛砖引玉的入门示例,帮助大家快速上手并对LLM领域产生更多的探索与创新。
diff --git a/README_en.md b/README_en.md
index 0a3ffcd..ebc1ac1 100644
--- a/README_en.md
+++ b/README_en.md
@@ -26,18 +26,17 @@
</div>
-* This open-source project aims to train a miniature language model **MiniMind** from scratch, with a size of just 26MB.
-* **MiniMind** is extremely lightweight, approximately $\frac{1}{7000}$ the size of GPT-3, designed to enable fast
- inference and even training on CPUs.
-* **MiniMind** is an improvement on the DeepSeek-V2 and Llama3 architectures. The project includes all stages of data
- processing, pretraining, SFT, and DPO, and features a Mixture of Experts (MoE) model.
-* This is not only the implementation of an open-source model, but also a tutorial for getting started with large
- language models (LLMs).
-* We hope that this project serves as a stepping stone for researchers and developers, providing an introductory example
- to help them quickly get started and foster more exploration and innovation in the LLM field.
-
- > To avoid any misunderstanding, "fastest 3 hours" refers to the requirement of using hardware with higher
- specifications than the author's setup. Detailed specifications will be provided below.
+* This open-source project aims to train a tiny language model called **MiniMind** from scratch in just 3 hours, with a model size of only 26.88M.
+
+* **MiniMind** is extremely lightweight, with the smallest version being approximately $\frac{1}{7000}$ the size of GPT3, making it possible for even an ordinary personal GPU to perform quick inference and even training.
+
+* **MiniMind** provides the full-stage code for a simplified large model structure, dataset cleaning and preprocessing, supervised pretraining, supervised instruction fine-tuning (SFT), low-rank adaptation (LoRA) fine-tuning, and direct preference alignment with reinforcement learning without rewards (DPO). It also includes code for expanding to sparse models with mixed experts (MoE) and multi-modal vision language models (VLM): [MiniMind-V](https://github.com/jingyaogong/minimind-v).
+
+* This is not just an implementation of an open-source model but also a tutorial for getting started with large language models (LLM).
+
+* We hope this project will serve as an introductory example for researchers, helping them quickly get started and inspiring more exploration and innovation in the LLM field.
+
+> To avoid misinterpretation, "fastest 3 hours" means you need a machine with hardware configuration superior to mine. Detailed specifications will be provided below.
---