Skip to content

🧠 两小时从 0 训出你的 64M 大模型。Train your own 64M LLM from scratch in 2 hours.

fix data_process bug

gongjyb4359b333577
diff --git a/README.md b/README.md
index d4f871e..ffaa032 100644
--- a/README.md
+++ b/README.md
@@ -687,6 +687,8 @@ minimind模型本身没有使用较大的数据集训练,也没有针对回答
 
<a href="https://github.com/chuanzhubin"><img src="https://avatars.githubusercontent.com/u/2813798" width="70px" height="70px"/></a>
&nbsp;
+<a href="https://github.com/iomgaa-ycz"><img src="https://avatars.githubusercontent.com/u/124225682" width="70px" height="70px"/></a>
+&nbsp;
## 😊鸣谢
diff --git a/README_en.md b/README_en.md
index 7c88224..e2e558a 100644
--- a/README_en.md
+++ b/README_en.md
@@ -756,6 +756,8 @@ your model with third-party UIs, such as fastgpt, OpenWebUI, etc.
&nbsp;
<a href="https://github.com/chuanzhubin"><img src="https://avatars.githubusercontent.com/u/2813798" width="70px" height="70px"/></a>
&nbsp;
+<a href="https://github.com/iomgaa-ycz"><img src="https://avatars.githubusercontent.com/u/124225682" width="70px" height="70px"/></a>
+&nbsp;
## 😊Thanks for
diff --git a/data_process.py b/data_process.py
index 9c03628..047ff0e 100644
--- a/data_process.py
+++ b/data_process.py
@@ -95,7 +95,7 @@ def process_seq_monkey(chunk_size=50000):
if doc_ids:
arr = np.array(doc_ids, dtype=np.uint16)
- with open(f'./dataset/clean_seq_monkey.bin', 'wb') as f:
+ with open(f'./dataset/clean_seq_monkey.bin', 'ab') as f:
f.write(arr.tobytes())