Skip to content

AI media harness for planning, generating, and editing locally

TypeScript

Enable WebGPU on WebKitGTK and log media features

Bitti1643736cfdff
diff --git a/docs/linux.md b/docs/linux.md
index 8ab2113..546d16d 100644
--- a/docs/linux.md
+++ b/docs/linux.md
@@ -47,17 +47,32 @@ the system linker does not.
## WebKitGTK
-Tauri uses WebKitGTK on Linux. Unlike WebView2, it ships WebCodecs and WebGPU
-behind runtime feature flags. `src-tauri/src/linux_webview.rs` turns them on
-(`WebCodecsVideoEnabled`, `WebCodecsAudioEnabled`, `WebCodecsAV1Enabled`,
-`WebGPUEnabled`) with GPU compositing and WebGL, and reloads the page once so
-the first document has them. The feature API needs WebKitGTK 2.42 or later; it is
-looked up at run time, so older versions still start and the diagnostic log
-names the missing features.
+Tauri uses WebKitGTK on Linux. Unlike WebView2, it ships WebGPU switched off
+(WebKitGTK 2.52 has WebCodecs on, but older releases do not), so
+`src-tauri/src/linux_webview.rs` turns on the `WebCodecsVideo`,
+`WebCodecsAudio`, `WebCodecsAV1` and `WebGPU` feature flags, with GPU
+compositing and WebGL, and reloads the page once so the first document has
+them. The feature API needs WebKitGTK 2.42 or later; it is looked up at run
+time, so older versions still start. The diagnostic log records the result
+(`app webview`, listing every GPU and codec flag when one is missing), and the
+frontend's `webview ready` entry records whether `VideoDecoder`,
+`VideoEncoder`, `AudioDecoder` and `navigator.gpu` exist.
WebKitGTK decodes and encodes through GStreamer. The .deb and .rpm depend on the
-base, good and bad plugin sets; `gstreamer1.0-libav` is recommended for H.264
-on systems without a hardware VA-API decoder. The AppImage bundles GStreamer.
+system's base, good and bad plugin sets (which include VA-API and NVIDIA
+hardware codecs) and recommend `gstreamer1.0-libav`, which the user's
+distribution provides.
+
+The AppImage bundles GStreamer, but only an allowlist of plugins:
+`scripts/linux/linuxdeploy-plugin-gstreamer.sh` replaces the plugin the Tauri
+CLI would download, which copies every plugin on the build machine, FFmpeg
+(gst-libav) and GPL encoders included. Hardware H.264/HEVC/AV1 goes through
+VA-API and NVIDIA's driver (`va`, `nvcodec`); VP8/VP9/AV1/Opus have software
+codecs. The same plugin removes NVIDIA cuBLAS, which linuxdeploy would otherwise
+copy in (over 500 MB) through `libslopfab.so`. After building,
+`package-linux-app.sh` fails if FFmpeg, x264/x265 or cuBLAS is in the AppImage.
+AAC audio needs a decoder the AppImage does not ship; the .deb and .rpm use the
+system's.
## Desktop integration
diff --git a/src-tauri/src/app.rs b/src-tauri/src/app.rs
index 523df0c..7b3d27c 100644
--- a/src-tauri/src/app.rs
+++ b/src-tauri/src/app.rs
@@ -84,8 +84,6 @@ pub fn run() {
let window = builder.build()?;
native_shell::round_corners(&window);
native_shell::disable_browser_behaviour(&window);
- #[cfg(target_os = "linux")]
- crate::linux_webview::configure(&window);
native_shell::watch_accent(app.handle());
match diagnostics::initialize(app.handle()) {
Ok(info) => diagnostics::info(
@@ -101,6 +99,9 @@ pub fn run() {
),
Err(error) => eprintln!("Could not initialize diagnostic logging: {error}"),
}
+ // After logging starts: it records which WebKit features came on.
+ #[cfg(target_os = "linux")]
+ crate::linux_webview::configure(&window);
app.state::<worker::Workers>().start(app.handle(), &app_paths::data_directory(app.handle()));
window::apply_theme(&window);
window.show()?;
diff --git a/src-tauri/src/linux_webview.rs b/src-tauri/src/linux_webview.rs
index 03916ed..6600819 100644
--- a/src-tauri/src/linux_webview.rs
+++ b/src-tauri/src/linux_webview.rs
@@ -13,13 +13,9 @@ use std::ffi::{c_char, c_int, c_void, CStr};
use tauri::Runtime;
use webkit2gtk::{glib::translate::ToGlibPtr as _, HardwareAccelerationPolicy, SettingsExt as _, WebViewExt as _};
-/// WebKit preference keys (UnifiedWebPreferences.yaml) the page relies on.
-pub(crate) const REQUIRED_FEATURES: &[&str] = &[
- "WebCodecsVideoEnabled",
- "WebCodecsAudioEnabled",
- "WebCodecsAV1Enabled",
- "WebGPUEnabled",
-];
+/// WebKitFeature identifiers the page relies on: the preference keys from
+/// UnifiedWebPreferences.yaml without their "Enabled" suffix.
+pub(crate) const REQUIRED_FEATURES: &[&str] = &["WebCodecsVideo", "WebCodecsAudio", "WebCodecsAV1", "WebGPU"];
type GetAllFeatures = unsafe extern "C" fn() -> *mut c_void;
type ListLength = unsafe extern "C" fn(*mut c_void) -> usize;
@@ -36,13 +32,16 @@ pub(crate) struct FeatureReport {
pub(crate) missing: Vec<&'static str>,
/// A flag was off until now, so a page already loaded does not have it.
pub(crate) changed: bool,
+ /// Every GPU and codec flag this WebKitGTK has, as `name=state`, so the
+ /// log shows what a renamed flag is called now.
+ pub(crate) related: Vec<String>,
}
/// The features from REQUIRED_FEATURES that `available` offers, and the rest.
pub(crate) fn match_features<'a>(available: impl IntoIterator<Item = &'a str>) -> FeatureReport {
let available: Vec<&str> = available.into_iter().collect();
let (enabled, missing) = REQUIRED_FEATURES.iter().partition(|name| available.contains(name));
- FeatureReport { enabled, missing, changed: false }
+ FeatureReport { enabled, missing, ..FeatureReport::default() }
}
/// Switches on the flags in REQUIRED_FEATURES that this WebKitGTK knows.
@@ -69,6 +68,7 @@ fn enable_features(settings: &webkit2gtk::Settings) -> Result<FeatureReport, Str
return Err("WebKitGTK returned no feature list.".into());
}
let mut available = Vec::new();
+ let mut related = Vec::new();
let mut changed = false;
for index in 0..length(list) {
let feature = get(list, index);
@@ -80,6 +80,9 @@ fn enable_features(settings: &webkit2gtk::Settings) -> Result<FeatureReport, Str
continue;
}
let name = CStr::from_ptr(name).to_string_lossy();
+ if name.contains("GPU") || name.contains("Codec") {
+ related.push(format!("{name}={}", get_enabled(settings_ptr, feature) != 0));
+ }
if let Some(required) = REQUIRED_FEATURES.iter().find(|required| **required == name) {
if get_enabled(settings_ptr, feature) == 0 {
set_enabled(settings_ptr, feature, 1);
@@ -89,7 +92,7 @@ fn enable_features(settings: &webkit2gtk::Settings) -> Result<FeatureReport, Str
}
}
unref(list);
- Ok(FeatureReport { changed, ..match_features(available) })
+ Ok(FeatureReport { changed, related, ..match_features(available) })
}
}
@@ -125,7 +128,7 @@ pub(crate) fn configure<R: Runtime>(window: &tauri::WebviewWindow<R>) {
"app",
"webview",
"This WebKitGTK lacks some media or GPU features. Update WebKitGTK for video decoding, export and effects.",
- serde_json::json!({ "enabled": report.enabled, "missing": report.missing }),
+ serde_json::json!({ "enabled": report.enabled, "missing": report.missing, "related": report.related }),
),
Err(error) => crate::diagnostics::warn("app", "webview", &error, serde_json::json!({})),
}
@@ -145,8 +148,8 @@ mod tests {
#[test]
fn reports_which_required_features_this_webkit_offers() {
- let report = match_features(["WebGPUEnabled", "WebCodecsVideoEnabled", "SomethingElse"]);
- assert_eq!(report.enabled, ["WebCodecsVideoEnabled", "WebGPUEnabled"]);
- assert_eq!(report.missing, ["WebCodecsAudioEnabled", "WebCodecsAV1Enabled"]);
+ let report = match_features(["WebGPU", "WebCodecsVideo", "WebGPUHDR"]);
+ assert_eq!(report.enabled, ["WebCodecsVideo", "WebGPU"]);
+ assert_eq!(report.missing, ["WebCodecsAudio", "WebCodecsAV1"]);
}
}
diff --git a/src/lib/diagnostics.ts b/src/lib/diagnostics.ts
index 1de5319..4a9f54a 100644
--- a/src/lib/diagnostics.ts
+++ b/src/lib/diagnostics.ts
@@ -62,8 +62,14 @@ export function installGlobalDiagnostics(): void {
window.addEventListener("unhandledrejection", (event) => {
writeDiagnostic("error", "webview", "unhandled_rejection", describeDiagnosticError(event.reason), errorContext(event.reason));
});
+ /* The media pipeline's engine features, which WebKitGTK only has when
+ src-tauri/src/linux_webview.rs managed to switch them on. */
writeDiagnostic("info", "webview", "ready", "Frontend diagnostics installed.", {
userAgent: navigator.userAgent,
+ videoDecoder: typeof globalThis.VideoDecoder === "function",
+ videoEncoder: typeof globalThis.VideoEncoder === "function",
+ audioDecoder: typeof globalThis.AudioDecoder === "function",
+ webGpu: "gpu" in navigator,
});
}