Skip to content

Distribute and run LLMs with a single file.

Add whisperfile server documentation

Alonso Silva4f1fae42d7f1
diff --git a/whisper.cpp/doc/index.md b/whisper.cpp/doc/index.md
index 17cb7e14..26cca747 100644
--- a/whisper.cpp/doc/index.md
+++ b/whisper.cpp/doc/index.md
@@ -11,3 +11,4 @@ by Georgi Gerganov, et al.
- [Packaging](packaging.md)
- [Using GPUs](gpu.md)
- [Speech Translation](translate.md)
+- [Server](server.md)
diff --git a/whisper.cpp/doc/server.md b/whisper.cpp/doc/server.md
new file mode 100644
index 00000000..c3e238be
--- /dev/null
+++ b/whisper.cpp/doc/server.md
@@ -0,0 +1,69 @@
+# whisper.cpp http server
+
+Simple http server. WAV Files are passed to the inference model via http requests.
+
+https://github.com/ggerganov/whisper.cpp/assets/1991296/e983ee53-8741-4eb5-9048-afe5e4594b8f
+
+## Usage
+
+```
+./whisperfile-0.9.0 -m whisper-tiny.en-q5_1.bin
+
+usage: ./whisperfile-0.9.0 -m whisper-tiny.en-q5_1.bin [options]
+
+options:
+ -h, --help [default] show this help message and exit
+ -t N, --threads N [4 ] number of threads to use during computation
+ -p N, --processors N [1 ] number of processors to use during computation
+ -ot N, --offset-t N [0 ] time offset in milliseconds
+ -on N, --offset-n N [0 ] segment index offset
+ -d N, --duration N [0 ] duration of audio to process in milliseconds
+ -mc N, --max-context N [-1 ] maximum number of text context tokens to store
+ -ml N, --max-len N [0 ] maximum segment length in characters
+ -sow, --split-on-word [false ] split on word rather than on token
+ -bo N, --best-of N [2 ] number of best candidates to keep
+ -bs N, --beam-size N [-1 ] beam size for beam search
+ -wt N, --word-thold N [0.01 ] word timestamp probability threshold
+ -et N, --entropy-thold N [2.40 ] entropy threshold for decoder fail
+ -lpt N, --logprob-thold N [-1.00 ] log probability threshold for decoder fail
+ -debug, --debug-mode [false ] enable debug mode (eg. dump log_mel)
+ -tr, --translate [false ] translate from source language to english
+ -di, --diarize [false ] stereo audio diarization
+ -tdrz, --tinydiarize [false ] enable tinydiarize (requires a tdrz model)
+ -nf, --no-fallback [false ] do not use temperature fallback while decoding
+ -ps, --print-special [false ] print special tokens
+ -pc, --print-colors [false ] print colors
+ -pr, --print-realtime [false ] print output in realtime
+ -pp, --print-progress [false ] print progress
+ -nt, --no-timestamps [false ] do not print timestamps
+ -l LANG, --language LANG [en ] spoken language ('auto' for auto-detect)
+ -dl, --detect-language [false ] exit after automatically detecting language
+ --prompt PROMPT [ ] initial prompt
+ -m FNAME, --model FNAME [models/ggml-base.en.bin] model path
+ -oved D, --ov-e-device DNAME [CPU ] the OpenVINO device used for encode inference
+ --host HOST, [127.0.0.1] Hostname/ip-adress for the server
+ --port PORT, [8080 ] Port number for the server
+ --convert, [false ] Convert audio to WAV, requires ffmpeg on the server
+```
+
+> [!WARNING]
+> **Do not run the server example with administrative privileges and ensure it's operated in a sandbox environment, especially since it involves risky operations like accepting user file uploads and using ffmpeg for format conversions. Always validate and sanitize inputs to guard against potential security threats.**
+
+## request examples
+
+**/inference**
+```
+curl 127.0.0.1:8080/inference \
+-H "Content-Type: multipart/form-data" \
+-F file="@<file-path>" \
+-F temperature="0.0" \
+-F temperature_inc="0.2" \
+-F response_format="json"
+```
+
+**/load**
+```
+curl 127.0.0.1:8080/load \
+-H "Content-Type: multipart/form-data" \
+-F model="<path-to-model-file>"
+```