Skip to content

armgit / Edge0

Public Imported from GitHub
0

Run 35B MoE models in ~2.5 GB of RAM. Weights stream from SSD; a trained router predicts loads a step ahead. Mac, iPhone, Android, Windows supported

Python
572 B
Raw Blame
"""Streaming expert offload: mmap stores, shared LRU caches, staged
decode slots, whole-layer prefill and hot-expert pins."""

from edge0.streaming.cache import PrefetchBuffer, SharedExpertCache
from edge0.streaming.install import install_streaming_experts
from edge0.streaming.layer import StreamingSwitchGLU
from edge0.streaming.mmap import SafetensorsMmap
from edge0.streaming.options import LayerOptions

__all__ = [
    "SafetensorsMmap",
    "SharedExpertCache",
    "PrefetchBuffer",
    "StreamingSwitchGLU",
    "LayerOptions",
    "install_streaming_experts",
]