37 tok/s from Qwen3-Next-80B (48 GB) on a 32 GB laptop, 24 without drafts vs 2.8 for llama.cpp; reading 62 vs 16 tok/s. Faster than llama.cpp on all 13 models compared in Q4_K_M. LLMs bigger than your RAM, pure C, CPU or GPU.
Serving the model
Janas as an MCP server
janas-mcp offers the model running here to the clients of the Model Context Protocol - Claude Code, editors, other agents - as tools: generate asks it a question (a prompt, and optionally a system message, a longest answer and a temperature) and returns its answer, and embed returns the vector of a text when an embedding model is given with --embedding-model. The client starts it and speaks to it over its standard input and output; it is configured like any other stdio server:
{"mcpServers": {"janas": {"command": "/path/to/janas-mcp",
"args": ["/path/to/qwen3-4b.jns"]}}}
It answers both eras of the protocol: a request that names revision 2026-07-28 in its _meta is served statelessly, and a client that opens with initialize gets the handshake of the revision it asks for (2025-11-25, 2025-06-18, 2025-03-26 or 2024-11-05). A ping is answered while a generation runs, and notifications/cancelled stops it. Every call starts a conversation of its own; the reasoning of a model that reasons is never returned, only the answer. janas-mcp --help lists the options (--ctx, --temp, --max, --think, --timeout and the memory ones).
What has been run: a probe script speaking each era to it with Qwen3-0.6B and Qwen3-Embedding-0.6B, janas-chat with Qwen3-4B calling its two tools through libjanas_mcp, and Claude Code (2.1.282, revision 2026-07-28), which listed both tools and called them - a translation, a summary, an embedding - with Qwen3-4B and Qwen3-Embedding-0.6B behind them. To add it there: claude mcp add janas -- /path/to/janas-mcp /path/to/model.jns.
Using it over HTTP
bin/x86_64-linux/janas-server qwen3-next.jns
janas-server puts the model behind an HTTP API that follows OpenAI's, on http://127.0.0.1:8080/v1, so that a client written for that API can use a model running on this machine. It takes the model options janas-chat takes (--ctx, --cache, --reserve, --mtp, --draft, --mode, --no-gpu, --prompt-on, --drafts-on, --reply-on, --npu-int8) and these:
| Option | |
|---|---|
--host ADDR |
the address to listen on; 127.0.0.1 (the default) is this machine only, 0.0.0.0 opens it to the network |
--port N |
8080 by default |
--api-key KEY |
ask every client for this key, as Authorization: Bearer KEY; JANAS_API_KEY sets it too |
--name ID |
the model's name for clients; the file's name without .jns by default |
--quality <low|medium|high> |
the lowest quality the engine may choose to be faster where memory is short: low (default) the experts at four bits where they do not all fit and the light ones not in memory left out, medium the four bits only, high nothing lowered (see the chat) |
--think on|off |
reasoning before replying, for models that do it, when a request does not say |
--reasoning-budget N |
the longest reasoning, in tokens, when a request does not say: there it is closed and the model answers (-1: no limit, the default; 0: no reasoning) |
--test |
serve a chat page for trying it out, on /test (below) |
--max-input N |
the longest input a request may have, in tokens, below the context: a longer one gets a 400 (input_limit_exceeded) whose error says input_tokens, max_input_tokens, excess_tokens and the parts, and nothing is ever cut to fit |
--warn-input N |
longer inputs are taken, and said on stderr with their parts |
--fim-prefix N, --fim-suffix N, --fim-max N |
completions with a suffix, an editor filling in code at its cursor: the tokens before the cursor read at most (2048; 0: all) and the lines after it (8; 0: all); --fim-max N the tokens written there at most, whatever the request asks (256; 0: as asked), since an editor asks for thousands and the requests of the next keystrokes wait for them. Over the budget the text before the cursor is cut at a line to three quarters of it, and the start of the window is kept across keystrokes, also when the editor slides its window, so that a keystroke reads only what it changed (see code completion) |
--metrics FILE, --progress S |
JSON lines of what every reply costs, and progress lines while a long prompt is read (see Watching a long prompt); --verbose prints progress every 30 s |
--tls-cert FILE, --tls-key FILE |
serve HTTPS with this certificate and key (PEM; below) |
--no-web |
refuse the web_search tool in /v1/responses (below) |
--code-interpreter |
run the code a model writes (code_interpreter, below), in a cage; off by default |
--code-interpreter-python PATH |
the python3 inside the cage, a path under /usr (the system's by default) |
--no-mcp |
refuse tools of type mcp in /v1/responses: no connection leaves the server on a request's word (below) |
--mcp-allow HOST[:PORT] |
an MCP server at a private or local address that requests may name all the same (again for more; [::1] for an IPv6 address): by default only public addresses are reached (below) |
--allow-origin ORIGIN |
a web page of another origin that may call the server, written as the browser sends it (http://localhost:3000; again for more; * for any), answered with the CORS headers it needs; pages of other sites are refused (below) |
--keep N |
conversations kept computed besides the one in use (8; 0: none), so that clients taking turns, or a client's requests on the side for titles and tags, do not have their conversations read again from the start |
--keep-disk GIB |
disk for them below the memory, in ~/.cache/janas, so that they outlive the server: a long system message and tools are read once (8; 0: none; never more than a quarter of the space left). Written only when a conversation leaves the memory and when the server stops |
--keep-memory GIB |
the memory for them; by default half of what the expert cache and the margin left to other programs leave free at start, at least 256 MiB |
--store DIR |
where stored completions, responses and conversations are kept, so that they outlive the server (~/.local/share/janas/server) |
--no-store |
keep them in memory only |
--queue N |
requests allowed to wait (16) |
--max-body MIB |
the largest request body (32) |
--connections N |
open connections at most (64) |
--verbose |
a line per request on the standard error |
What it answers:
| Request | |
|---|---|
GET /v1/models, GET /v1/models/{model} |
the models it runs, with a description of how they run here; DELETE answers that a model read from a file is not deleted through the API |
POST /v1/chat/completions |
a conversation, the reply whole or streamed ("stream": true, server-sent events), with tools, JSON output, log-probabilities and more than one reply (below) |
GET, POST, DELETE /v1/chat/completions/{id}, GET /v1/chat/completions, .../messages |
the completions a client asked to keep ("store": true): read, listed with their metadata, changed, deleted |
POST /v1/completions |
raw text to continue, no chat format around it; several prompts, token ids, echo, logprobs, best_of, suffix |
POST /infill |
llama.cpp's route for filling in code, which editors' extensions made for llama.cpp ask (llama.vscode): input_prefix, prompt, input_suffix, n_predict (0: the prompt only, read for the next request), n_indent, t_max_predict_ms, the answer as llama.cpp's (content, timings); input_extra is not used, and the most likely token is always taken |
POST /v1/responses and the six operations under it |
OpenAI's newer API: items in and out, previous_response_id, conversations, streamed events, background and cancel, input_items, input_tokens, compact |
/v1/conversations and its seven operations |
conversations the server keeps, and their items |
/v1/files and its four operations |
files a client uploads (purpose, expires_after), listed, read, downloaded and deleted (below) |
/v1/uploads and its three operations |
a file sent in parts, joined into a File (below) |
/v1/batches and its three operations |
a file of requests run in the background (below) |
/v1/vector_stores and its operations |
stores of files cut into chunks and searched (below) |
POST /v1/embeddings |
with --embedding-model FILE: the vectors of texts (below) |
POST /v1/audio/transcriptions, /v1/audio/translations |
with --speech-model FILE: speech to text with a Whisper model (below) |
POST /v1/moderations |
asked of the chat model (below) |
GET /health |
{"status":"ok"} |
curl http://127.0.0.1:8080/v1/chat/completions \
-H 'Content-Type: application/json' \
-d '{"messages": [{"role": "user", "content": "What is the capital of Sardinia?"}]}'
A client that lets you set the address of an OpenAI-compatible API should need nothing else: the base URL is http://127.0.0.1:8080/v1, and the key whatever you gave --api-key (anything, if you gave none). It has been tried here with curl, its own test page, the openai Python SDK and Open WebUI 0.11.4 (driven through its own API: streamed chat, titles, tags, follow-up suggestions, and a document indexed with Janas-LLM's embeddings and asked about), and by a user with AnythingLLM.
Models that reason, and the limits a client sets. Qwen3 and its successors reason before they answer, unless told not to. OpenAI's max_completion_tokens (and max_output_tokens in the Responses API) counts the reasoning too, and janas-server takes it so. The older max_tokens of a chat, which OpenAI does not take at all for the models that reason, is taken for the answer alone: a client that sends it means the answer, and counted with the reasoning a small limit was spent before a word of it was written (Open WebUI asks for a chat's title with 1000, and got none). The reasoning itself has a ceiling only when one is asked for: reasoning_budget_tokens (or thinking_budget_tokens, the names llama.cpp takes) in a request, --reasoning-budget for the requests that say nothing. A model still reasoning there has its reasoning closed for it, a sentence ("I have thought enough; now I give the answer.") and the closing mark written as if it had written them, and goes on with its answer, which the limits above still bound: -1 means no ceiling, 0 no reasoning at all. Without one a model that never closed its reasoning would write until the context is full; issue #21's did, its arithmetic broken by a defect since fixed.
With Open WebUI, start the server with --think off: the titles, tags and suggestions it asks for after every reply are then quick, and a model that reasons can still be asked to (reasoning_effort). Its built-in tools, when they are on, add about 5,000 tokens to every prompt; the server reads them once and keeps them computed, but the first reply of a conversation waits for them.
The conversation is the client's. OpenAI's API keeps no state: every request carries the whole conversation. janas-server keeps the one it computed last, and reads again only what a request adds to it: a client that sends the same messages plus a new one pays for the new one alone, and usage.prompt_tokens_details.cached_tokens says how much was already computed. The replies it wrote are remembered with their reasoning, so the answer a client sends back - without the reasoning, as clients do - still counts as the same turn. A conversation that differs earlier is read again from where it differs, and from the start on models with a recurrent state (Qwen3-Next, Qwen3.5 and 3.6).
Reasoning comes apart from the answer, in reasoning_content, whole or streamed. "reasoning_effort": "none" turns it off for a request, any other effort turns it on, and "chat_template_kwargs": {"enable_thinking": false} works too; --think decides for requests that say nothing.
Beyond OpenAI's fields, which clients ignore: top_k and min_p in a request, and in every reply's usage a janas object with the context used and its size and the seconds spent reading the prompt and writing the reply.
Tools. tools, tool_choice (none, auto, required, a function) and parallel_tool_calls, and the older functions. The tools are written into the system message the way the model's chat template writes them, and a call the model opens is held to a grammar of the functions and their parameters until it closes it, so its arguments are always a JSON object valid against the function's schema. The calls come back in tool_calls, whole in one delta of the stream, and the reply ends with "finish_reason": "tool_calls"; sent back with the tools' answers, they are found again among the replies the server remembers, so the next turn reads only the answers. Qwen models write calls in one of two ways, and the template says which: as JSON (Qwen3, Qwen3-Next) or as XML (Qwen3.5 and 3.6, Qwen3-Coder). Both have been run here: the JSON way on Qwen3-4B and Qwen3-0.6B, the XML way on Qwen3.6-35B-A3B (two calls at once, their answers sent back, 422 of 471 prompt tokens reused on the next turn) and on Qwen3-Coder-Next, which describes the tools in a way of its own (two calls at once with valid arguments, their answers used in the reply, 496 of 562 prompt tokens reused).
MCP servers in the Responses API. A tool of type mcp (server_label, server_url, headers, authorization, allowed_tools, require_approval) has the server reach that MCP server over HTTP or HTTPS, as OpenAI's does. The response begins with an mcp_list_tools item (once per conversation), the model sees the tools as server_label__tool, and a call it writes either becomes an mcp_approval_request that ends the response - require_approval is always by default, as at OpenAI - and runs when the next request answers it with an mcp_approval_response, or, where no approval is needed, runs at once as an mcp_call item with its output, and the model goes on in the same response (up to eight rounds). Streamed, the calls come as response.mcp_call.* events. allowed_tools and the approval filters take names or {tool_names, read_only}; tool_choice may name an MCP tool. Connectors and tunnels are not served. A request makes the server open connections to the address it names, so it reaches only public ones: every address the name resolves to is checked, and the connection goes to one that passed, so a name that resolves to a public address when checked and to a private one when used gains nothing. Refused are loopback (127/8, ::1), private (10/8, 172.16/12, 192.168/16, fc00::/7), shared (100.64/10), link-local (169.254/16, where clouds answer with their machines' credentials, and fe80::/10), multicast, documentation, benchmarking and the other reserved ranges, and IPv4 addresses inside IPv6 ones (mapped, NAT64, 6to4) are judged as themselves; the error says not a public address. An MCP server of your own on this machine or the local network is named with --mcp-allow (--mcp-allow localhost:3001, --mcp-allow intranet.lan: the host as the request writes it, with or without a port). --no-mcp turns the connections off altogether, for a server others can reach. Tried with Qwen3-4B against the test server of tests/mcp_fake_server.c: a call run at once, one approved on the next request, and the same streamed; and through the openai SDK against DeepWiki over HTTPS, as OpenAI's own examples do: its tool listed, called, and the model answering from what it returned.
File search in the Responses API. A tool of type file_search (vector_store_ids, max_num_results up to 50, six by default since every passage goes into the model's context, filters, ranking_options.score_threshold) gives the model a search of those vector stores: it writes the query, the server searches every store and gives it the best passages, numbered on from any source found before. The response holds a file_search_call item with the query and the results (file, score, text, attributes), and the text file_citation annotations (file, name, index) for the [n] the model wrote. tool_choice: {"type": "file_search"} makes it search first. With Qwen3-4B and three files: "Which cheese goes into spaghetti alla carbonara?" searched, found the right file, answered "pecorino romano ... [1]", [1] a citation of that file.
Web search in the Responses API. A tool of type web_search (or its dated and preview names) gives the model a search of Wikipedia, the open source the project's services use: the model writes the query and the language, the server asks that Wikipedia (the English one when the first has nothing) and gives the model the best five pages' first sentences, numbered. The response holds a web_search_call item with the query and the pages' addresses, and the text the url_citation annotations of the [n] the model wrote after the facts. tool_choice: {"type": "web_search"} makes it search first; --no-web refuses the tool. Wikipedia is not news: the tool tells the model so. With Qwen3-4B: "Who designed the Eiffel Tower?" answered "Gustave Eiffel ... 1889 [3]", [3] annotated with the page; in Italian, streamed, "Eleonora d'Arborea" searched on the Italian Wikipedia and answered from it with [1]. The stream gives the search as an output item added and done, not yet as OpenAI's response.web_search_call.* events.
JSON output. response_format json_object or json_schema (and text.format in the Responses API): the answer is held to a grammar token by token, so it is valid JSON when the model ends it (not when max_tokens cuts it). A schema is held to its types, the properties of an object in the schema's order with the required ones always there and no others, array items and bounds up to 32, string lengths up to 4,096, enum, const, anyOf, oneOf, $ref within the schema and nullable. Integers are held to minimum, maximum, exclusiveMinimum, exclusiveMaximum and multipleOf (up to 50) exactly, digit by digit, so a number that cannot end inside the bounds is never begun; other numbers to their bounds without an exponent, with a fraction only where no fraction can cross a bound (up to 10 writes 10, never 10.5; from 1.5 writes 2 and more). The formats date, time, date-time, uuid, email and ipv4 are held as their grammar (a date's month 01 to 12 and day 01 to 31, a time with its offset, each octet of an address 0 to 255). A string's pattern is held as its grammar too: characters and classes ([a-z], [^,], \d, \w, \s, .), groups, alternatives and repetitions (*, +, ?, {m,n} up to 256), found anywhere in the string unless anchored with ^ and $ as JSON Schema says; a quote, a backslash or a control character the pattern asks for is written escaped. A pattern with anchors inside, back references, look-arounds or Unicode classes is not enforced (the string is left free), nor are minLength and maxLength beside a pattern or a format. The other formats and allOf of more than one schema are not enforced. The reasoning, where there is one, stays free.
Log-probabilities. logprobs and top_logprobs (up to 20) in chat, logprobs in completions, top_logprobs in responses. They are the model's own: where a grammar forced a token the model found unlikely, its log-probability says so.
More than one reply with n (and best_of in completions, which keeps the best by mean log-probability); presence_penalty, frequency_penalty and logit_bias as OpenAI defines them. The replies of n are written one after the other on the same prompt.
Embeddings come from a second, small model opened with --embedding-model, such as Qwen3-Embedding-0.6B: texts or token ids, one or many, dimensions to cut the vector shorter, floats or base64. Against llama.cpp on the same file the vectors of the same text agree to a cosine of 0.98 to 0.996, and the similarities between texts come out alike.
Speech to text comes from a Whisper model opened with --speech-model (converted by hf2jns_whisper: models.md), beside the chat model and apart from its queue, one recording at a time: POST /v1/audio/transcriptions writes down what is said in the language it is said in (found out, or language), POST /v1/audio/translations into English, as OpenAI's whisper-1 answers - file (every format OpenAI lists - flac, m4a, mp3, mp4, mpeg, mpga, oga, ogg, wav, webm - and a few more, at any rate and with any channels, read by Janas's own decoders: WAV, FLAC, MP3 and the other MPEG audio, Ogg with Vorbis, Opus or FLAC, MP4/M4A/MOV with AAC (LC, Main, LTP, HE-AAC v1 and v2), ALAC, MP3, Opus, FLAC or PCM, AAC in ADTS frames, WebM/Matroska with Opus, Vorbis, FLAC, AAC, MPEG audio, ALAC or PCM; the encoder's delay and padding trimmed as the container says; AAC's low-delay and USAC kinds are refused with a 400 that says so), model (any name: there is one speech model), prompt (words and spelling to expect, or the text said before), temperature, response_format json, text, srt, vtt or verbose_json (the language, the duration and the segments with their times, tokens, average log-probability, probability of silence and compression ratio - the text's bytes over their zlib-compressed bytes, to the byte as zlib makes them, by Janas's own deflate - which, above 2.4, has the window decoded again at a higher temperature, as OpenAI's Whisper does), timestamp_granularities[] segment and word with verbose_json (the words with their start and end, found as OpenAI's open-source Whisper finds them: the alignment heads' attention over the sound, its path by dynamic time warping, punctuation joined to its word, the next window starting at the last word's end), chunking_strategy (auto, or {"type": "server_vad", "threshold", "prefix_padding_ms", "silence_duration_ms"}: only the stretches of speech Silero VAD finds are read, the windows within each of them, the text before carried on; silero-vad.jns beside the speech model, or --vad-model), response_format diarized_json with known_speaker_names[] and known_speaker_references[] (data: URLs of a few seconds of each; see below), include[]=logprobs (each text token's log-probability and bytes, with json), stream (server-sent events: a transcript.text.delta for each segment as soon as it is final, then transcript.text.done with the whole text and the usage; a client that goes away stops the transcription at its next window), usage in seconds. Who speaks when (diarized_json, as OpenAI's gpt-4o-transcribe-diarize): segments of type transcript.text.segment, each a speaker's turn with its start, end, text and speaker - a known speaker's name, or A, B... in the order they first speak - also streamed as transcript.text.segment events once the recording is read; the speakers found by WeSpeaker's ResNet34 embeddings over windows of 1.5 s of the speech and WeSpeaker's spectral clustering, each word given the speaker of the windows over it; wespeaker-resnet34.jns beside the speech model, or --speaker-model. Tried with the openai SDK on a meeting of 50 seconds spoken by three synthetic voices: ten turns of ten, the three speakers found, the two known ones named. Tried with curl and the openai SDK and whisper-small: John F. Kennedy's 11 seconds word for word in one segment, 46 seconds of Italian in ten segments with their times (the same as whisper.cpp's but for two words, a comma and one time 40 ms apart) and translated into English, srt, vtt, a text file and an unknown language refused; with the openai SDK again, word times, log-probabilities, chunking by voice activity and an M4A of 46 seconds streamed.
Moderations are asked of the chat model: at temperature 0 it answers OpenAI's thirteen categories with a JSON object held to a schema, and each score is the chance it gave true against false in that place. It is a general model following instructions, not a classifier trained for the task.
Kept on disk. Stored completions, responses and conversations are written, one file each, to ~/.local/share/janas/server (or --store DIR), and read back when the server starts again; the oldest go past a thousand of each, and past 1 GiB of them all, whatever their kind; a document is at most 64 MiB, so a conversation that would grow past it gets a 400 (conversation_too_large). The directory and its files are readable by their owner alone, since they hold what the clients said, and one server at a time uses a directory. --no-store keeps them in memory only, until the server stops. A response running in the background when the server is killed is found as failed at the next start. Sound in messages is refused with a 400 that says so; pictures are read by the models that see (below).
Files. POST /v1/files takes an upload as the SDKs send it (multipart/form-data: file, purpose - assistants, batch, fine-tune, vision, user_data or evals - and expires_after from an hour to 30 days; a batch's files expire after 30 days, as at OpenAI). The bytes go to the store's directory (.files in it, readable by its owner alone), each under its id; the name the client gave is shown, never used as a path. Without --store, or with --no-store, they go to a directory of their own under TMPDIR, removed when the server stops. An upload is at most --max-body (32 MiB unless raised); all the files together at most 16 GiB, and a gigabyte of disk is always left free. GET /v1/files lists them newest first (purpose, order, limit, after), GET /v1/files/{id}/content gives the bytes back. Tried with the openai SDK: an upload with binary bytes read back the same, an expiry, the list by purpose and in pages, a delete then a 404, a wrong purpose a 400.
Uploads, for a file larger than one request: POST /v1/uploads announces it (filename, purpose, bytes up to 8 GiB, mime_type), POST /v1/uploads/{id}/parts sends a part of up to 64 MiB (whatever --max-body says), POST /v1/uploads/{id}/complete names the parts in order and joins them into a File, checked against the size announced and the md5 when given (a wrong one is a 400, and the upload stays open to try again); cancel drops it. An upload lasts an hour. Its parts wait beside the files; a server that restarts forgets what was pending. The openai SDK's uploads.upload_file_chunked sent 150 MiB in three parts with their MD5 and got the same bytes back.
Batches. A JSONL file of requests (uploaded with purpose batch, up to 50,000 lines and 200 MB, each {"custom_id", "method": "POST", "url", "body"}) to /v1/chat/completions, /v1/responses, /v1/completions, /v1/embeddings or /v1/moderations is checked first (a line that is not JSON, a url other than the batch's endpoint, a custom_id twice: failed, with the lines in errors), then run one request at a time behind every request a client is waiting for: a client never waits for more than the one line running (a chat reply took 0.12 s while a batch ran). Each line goes through the same code as the request it stands for; the answers go to the output file, the requests answered with an error to the error file, both Files with purpose batch_output once the batch ends (30 days, or output_expires_after), and request_counts and usage follow. A batch is saved after every line: stopped with the server, it goes on from there at the next start. cancel stops it after the line running, keeping what was done; past its 24 hours it ends expired the same way. A line with "stream": true is refused. Tried with the openai SDK on Qwen3-0.6B: a batch with two bad lines, an invalid file, a cancel halfway (13 of 30 kept), a server stopped after 3 lines of 12 and started again (12 answers, none twice).
Vector stores. A store holds files uploaded here (text: plain, Markdown, code, JSON, CSV; up to 32 MiB each; a file that is not text is failed with unsupported_file), each cut into chunks of about the tokens its chunking strategy says (auto: 800 tokens overlapping by 400; static from 100 to 4,096) at whitespace. With --embedding-model each chunk gets its embedding, made in the background behind every client's request, and a search compares a query's (asked as a query, as Qwen3-Embedding wants) with all of them by their cosine; without one, the search reads the words (BM25), so a store works on any machine. POST /v1/vector_stores/{id}/search takes a query or several, max_num_results (up to 50), filters on the files' attributes (eq, ne, gt, gte, lt, lte, in, nin, and, or) and ranking_options.score_threshold, and gives the chunks best first with a score from 0 to 1. Stores, their files and attributes are listed, read, changed and deleted as at OpenAI, and kept beside the files (.vectors); a batch of files (file_batches, up to 2,000, with file_ids or files each with its own attributes) is attached at once, followed by its counts, cancelled (the files still waiting are), and listed. Tried with the openai SDK, Qwen3-Embedding-0.6B and without: three files of different subjects, each question found its own file first (0.64 to 0.70 with embeddings, the others below 0.23), a binary file refused, the filters, the lists, a rename, a delete.
Evals. An eval names the shape of its data (data_source_config, custom with an item_schema) and its graders (testing_criteria): string_check (eq, ne, like, ilike), text_similarity (fuzzy_match, bleu, gleu, meteor, rouge_1 to rouge_5, rouge_l, cosine over word counts, with pass_threshold), and label_model and score_model, which the server's own model answers, held by a JSON schema to one of the labels or to a number in range; python graders are refused. A run takes the rows in the request (file_content) or in a JSONL file uploaded here (file_id), up to 10,000; with a completions or responses data source the model writes each row's sample from input_messages (a template filled with {{item.field}}, or an item_reference), with a jsonl one the rows bring their own sample. The graders read {{item.…}} and {{sample.output_text}}. A run goes on in the background behind every client's request, row by row: its result_counts and per_testing_criteria_results grow as it goes, each row's output item (sample, each grader's score, pass or fail) is kept beside the files (.evals) and listed by status; a run is saved after every row, so it goes on after a restart, and cancel stops it after the row it is on. Tried with the openai SDK and Qwen3-4B: three questions answered and graded by all four kinds of grader (the model as judge said correct and scored 10 of 10); a wrong expected answer judged incorrect; a run cancelled after 2 rows of 8; a server stopped after 2 rows of 5 and started again (5 output items, none twice).
Assistants. The Assistants API (/v1/assistants, /v1/threads, their messages, runs and run steps: 23 operations), which OpenAI is retiring in favour of the Responses API, is served for the programs written for it. An assistant keeps its instructions, its tools (functions and file_search) and its tool_resources (vector stores of this server, given or made of file_ids); a thread keeps its messages, and the files attached to a message for file_search go into the thread's own vector store. A run goes on in the background: the thread's messages (the last ones only with truncation_strategy), the instructions, the tools and the sampling (the run's, else the assistant's) go to the same code as /v1/responses; a search becomes a tool_calls step with its results, the answer a message of the thread with its citations as file_citation annotations, and calls of functions make the run wait (requires_action, ten minutes) for submit_tool_outputs, after which it goes on. cancel stops the model at its next token; a run is saved after every pass, so one cut by a stop goes on at the next start. With stream: true (creating a run, a thread and run, or submitting tool outputs) the run's events come as OpenAI sends them - thread.run.*, thread.run.step.*, thread.message.created, thread.message.delta as the model writes, thread.message.completed, then done when the run ends or waits for tool outputs - so the SDKs' event handlers work. With --code-interpreter, an assistant's code_interpreter runs in the thread's own container, kept from run to run with its variables, with the files of the assistant's and the thread's code_interpreter resources (and those attached to messages for it) in /mnt/data; a step shows the code, its logs and its images, the images made Files (purpose assistants_output), and a file the answer names becomes a file_path annotation. Pictures in the messages are read as in the Responses API (below). Tried with the openai SDK and Qwen3-4B: a question, a function called with the right arguments and its output used in the answer, a file attached to a message searched and cited, a cancel in 2 s, a run cut by a stop and finished after the restart; streamed, three sentences came in 50 pieces as written, and the function and the search the same way.
Pictures in messages. A model that sees - Qwen3.5 and Gemma 4 for now, with its vision encoder beside it (below, and models.md) - reads the pictures of the user's messages: image_url parts in chat completions ({"url": ...}) and input_image parts in the Responses API (an image_url, or the file_id of a file uploaded here), the url a data: URL (base64) or an http(s) one, which the server fetches at public addresses only (as the MCP servers, --mcp-allow naming the exceptions), 16 MiB at most a picture and 16 a request; PNG and JPEG, read by Janas's own decoders, the same pixels as Pillow. A picture of more than a megapixel is made smaller (Pillow's bicubic, as the model's processor does), so it is at most 1024 tokens (JANAS_IMAGE_MAX_TOKENS changes it; Gemma 4 at most 280, as its processor). A model without an encoder answers 400 with the reason; a picture in a message other than the user's, or bytes that are not a picture, get a 400 too. With Qwen3.5-0.8B through the openai SDK: a picture of a red circle and a blue square, "I see a red circle and a blue square." (217 tokens of prompt, under 5 s), the same picture by its file_id and by a data: URL in the Responses API, a URL at 127.0.0.1 refused.
Code interpreter. Off unless the server is started with --code-interpreter: the code a model writes then runs on this machine, so it is the choice of who runs the server. The code runs in a cage Janas builds itself (src/common/sandbox.c, Linux, without privileges: user namespaces, which most distributions allow): no network at all, no other process to see or start (threads only), none of your files - the root holds the system's /usr read-only, a few files of /etc the programs need, a /tmp of 256 MiB in memory and the container's own directory at /mnt/data - limits on memory, file sizes and open files, no privileges and a seccomp filter on the system calls that could reach the kernel's less guarded parts; whatever the code leaves in /mnt/data (a link to one of your files, say) is never followed by the server. Each container has one Python session, as a notebook: variables and imports stay from one piece of code to the next, the value of the last expression is shown, figures drawn with matplotlib are saved into /mnt/data (unless the code saved an image itself); a piece of code that runs past two minutes ends the session. The libraries are the system's (with Debian's packages: numpy, scipy, sympy, matplotlib, Pillow). In /v1/responses the tool code_interpreter (container an id, or {"type": "auto", "file_ids", "memory_limit"}) gives the model a function to run Python, run by the server: a code_interpreter_call item holds the code and its outputs (logs, and images as the URL of the container's file), and a file the answer names becomes a container_file_citation. The Containers API (/v1/containers, nine operations) makes containers (file_ids copied in, memory_limit 1g to 64g, expires_after up to 20 minutes since the last use), lists, reads and deletes them, and uploads, lists, reads, downloads and deletes their files; an expired container's session ends and its files go. Tried with the openai SDK and Qwen3-4B: the 30th Fibonacci number computed, a CSV uploaded and summed (the model corrected its code after two tracebacks), a chart drawn and downloaded as a PNG, a variable kept from one response to the next, links to /etc/passwd and /home planted by the code never listed nor served; and in tests/test_sandbox.c the cage against code that wants out (your files, writing outside, the network, other processes, a fork, 3 GiB of memory, a 16 MiB file, a full /tmp, an endless loop), with mutations of the cage caught.
Every other operation of OpenAI's API - there are 345 in the version it follows (2.3.0), 100 of them served - has a route that answers 501 and says why: not written yet, needs a model of another kind (sound, images, video), trains models, or belongs to the administration of OpenAI's service. The table is generated from the specification by tools/openapi_routes, which also counts what is done.
One request at a time. The model holds one sequence, so requests wait in line and run in the order they came; when the line is full the answer is 429. While a streamed request waits, the stream says where it stands in the line (: queue N, a comment clients skip) and the test page shows it. A client that closes the connection while a reply streams stops the reply.
On the network. It listens on this machine alone unless --host says otherwise, and then it warns if there is no key. Across a network, serve HTTPS: --tls-cert cert.pem --tls-key key.pem (PEM, the certificate's chain after it, the key not encrypted; RSA or EC) makes the server speak HTTPS alone on its port, TLS 1.2 and 1.3 through the Mbed TLS it carries, and checks at start that the key is the certificate's. Every request, stream and limit is the same as in clear. Not on Windows yet. Tried with a certificate of its own: curl and the openai SDK (models, a chat, a stream, four requests at once), a request in clear refused, an untrusted certificate refused by the client, a key that is not the certificate's refused at start.
Web pages. A page of any site you visit can make your browser send requests to 127.0.0.1: it cannot read the answers, but the model would still run and a tool of type mcp would still connect, and a site that makes its own name resolve to 127.0.0.1 would read them too. The browser says which page is asking, in the Origin header, and the server refuses every page but its own (/test, at an address or at localhost, this machine's name or the --host given) with a 403 (origin_not_allowed), key or no key. Programs send no Origin and are not concerned. A web client served from another origin - a chat on http://localhost:3000 that calls the server from the browser - is let in with --allow-origin http://localhost:3000, and gets the CORS headers it needs (OPTIONS answered too); --allow-origin '*' lets in any page, which undoes the protection.
The test page. With --test, http://127.0.0.1:8080/test is Janas-Chat Web: a chat in the browser that talks to the API like any client, with the reasoning in grey, a line under each reply with the context used and the speed (last, lowest, highest and mean of the conversation), and the settings - system message, temperature, reasoning, key - behind Settings. It reads the marks a model writes as janas-chat does, and also quotes, tables, struck text and links; a link asks before it opens, because links an AI writes can be wrong or unsafe. Markdown sample shows a text of ours with every mark it knows, without asking the model. The page is inside the program, loads nothing from anywhere else and may talk only to this server.