Skip to content

Automation foundation model for tiny devices: 2-bit, 8-29 MB, tool calls, ASR, structured extraction and embeddings on phones, wearables, smart homes, robots, cars and microcontrollers.

fix bugs

HenryNdubuakue146239056a4
diff --git a/src/tokenize_data.py b/src/tokenize_data.py
index cf66bc0..8c2ce15 100644
--- a/src/tokenize_data.py
+++ b/src/tokenize_data.py
@@ -23,7 +23,6 @@ from .data import (
prepare_tool_call_pairs,
train_tokenizer,
)
-from .gcs import upload_tokenized_data, upload_tokenizer
def _clear_local_caches():
@@ -69,6 +68,7 @@ def tokenize(args):
max_enc_len, max_dec_len)
try:
+ from .gcs import upload_tokenized_data, upload_tokenizer
upload_tokenizer(TOKENIZER_DIR)
upload_tokenized_data(CACHE_DIR)
except Exception as e: