Skip to content

armgit / Edge0

Public Imported from GitHub
0

Run 35B MoE models in ~2.5 GB of RAM. Weights stream from SSD; a trained router predicts loads a step ahead. Mac, iPhone, Android, Windows supported

Python
517 B
Raw Blame
"""edge0 serving layer: OpenAI-compatible chat endpoint over one engine."""

from edge0.server.chat import (ChatMessage, ChatRequest, ChatSession,
                               QueueServer, decode_tokens, parse_chat_request,
                               sse_format)
from edge0.server.app import (create_app, run_server, run_stdlib)

__all__ = [
    "ChatMessage", "ChatRequest", "ChatSession", "QueueServer",
    "decode_tokens", "parse_chat_request", "sse_format",
    "create_app", "run_server", "run_stdlib",
]