Automation foundation model for tiny devices: 2-bit, 8-29 MB, tool calls, ASR, structured extraction and embeddings on phones, wearables, smart homes, robots, cars and microcontrollers.
Update simple_attention_networks.md
Henry Ndubuakuffb1c5144c5a
diff --git a/docs/simple_attention_networks.md b/docs/simple_attention_networks.mdindex 0694917..c7332aa 100644--- a/docs/simple_attention_networks.md+++ b/docs/simple_attention_networks.md@@ -1,7 +1,7 @@# Simple Attention NetworksExperiments at Cactus showed that MLPs can be completely dropped from transformer networks, as long as the model relies on external knowledge source.-Function calling relies on external tools list, so we designed a simple attention network for function calling and distilled Gemini-3.1-Flash-Lite.+Function calling relies on external tools list, so we designed a simple attention network for function calling.```d=512, 8H/4KV, BPE=8192