Skip to content

Automation foundation model for tiny devices: 2-bit, 8-29 MB, tool calls, ASR, structured extraction and embeddings on phones, wearables, smart homes, robots, cars and microcontrollers.

fix(tokenizer): drop a CLI hint for a command that does not exist (#116)

Wu Shuwenaaf62b4168d2
diff --git a/needle/model/tokenizer.py b/needle/model/tokenizer.py
index 195599e..83d7a37 100644
--- a/needle/model/tokenizer.py
+++ b/needle/model/tokenizer.py
@@ -108,6 +108,6 @@ def get_tokenizer(vocab_size=None):
except Exception as e:
raise RuntimeError(
f"No pretraining tokenizer at {model_path} and HF download failed ({e}). "
- f"Run `needle tokenizer-train` (add --upload to share it via HF hub)."
+ f"It ships in the {HF_REPO} Hugging Face repo; retry once it is reachable."
) from e
return SANTokenizer(model_path)