Skip to content

🧠 两小时从 0 训出你的 64M 大模型。Train your own 64M LLM from scratch in 2 hours.

[update] readme

jingyaogong9348fde74344
diff --git a/README.md b/README.md
index 93091a6..39b4184 100644
--- a/README.md
+++ b/README.md
@@ -1218,7 +1218,7 @@ python train_agent.py
> MiniMind 在 Agentic RL 训练阶段的优化走势
-这里顺带提一下 `rollout_engine`。所谓“训推分离”,就是把 **参数更新** 和 **轨迹展开** 拆开:训练侧负责优化 policy,rollout 侧负责高吞吐采样,对上统一表现为“给我 prompt,我返回 rollout 结果;训练完以后,再把新权重同步回来”。因此训练脚本并不需要关心底层到底是本地 `generate` 还是远端 `inference` 引擎。
+这里顺带提一下 `rollout_engine`。所谓“训推分离”,就是把 **参数更新** 和 **轨迹展开** 拆开:训练侧负责优化 policy,rollout 侧负责高吞吐采样,对上统一表现为“给我 prompt,我返回 rollout 结果;训练完以后,再把新权重同步回来”。因此训练脚本并不需要关心底层到底是本地 `generate` 还是远端 `inference` 引擎。需要说明的是,当前实现仍是**同步**模式(采样完一批再更新),还不是纯 rollout buffer 的异步训练。
![rl-structure](./images/rl-structure.jpg)
diff --git a/README_en.md b/README_en.md
index 3b872b5..e8fffd8 100644
--- a/README_en.md
+++ b/README_en.md
@@ -1217,7 +1217,7 @@ python train_agent.py
> MiniMind optimization trends during the Agentic RL training stage
-Here I'll also briefly mention the `rollout_engine`. The so-called "training-inference separation" means decoupling **parameter updates** and **trajectory rollout**: the training side handles policy optimization, while the rollout side handles high-throughput sampling. From the top level, they uniformly present as "give me a prompt, I'll return rollout results; after training is done, sync the new weights back." Therefore, the training script doesn't need to care whether the underlying implementation is local `generate` or a remote `inference` engine.
+Here I'll also briefly mention the `rollout_engine`. The so-called "training-inference separation" means decoupling **parameter updates** and **trajectory rollout**: the training side handles policy optimization, while the rollout side handles high-throughput sampling. From the top level, they uniformly present as "give me a prompt, I'll return rollout results; after training is done, sync the new weights back." Therefore, the training script doesn't need to care whether the underlying implementation is local `generate` or a remote `inference` engine. Note that the current implementation is still **synchronous** (sample a batch, then update), not yet asynchronous training with a pure rollout buffer.
![rl-structure](./images/rl-structure.jpg)