A lightweight WebAssembly runtime that is fast, secure, and standards-compliant
//! Centralized definitions of the various `VM*` types whose layout is shared
//! between the runtime (which uses the actual structures) and the compiler
//! (which uses the types' offsets and has per-type alias regions).
//!
//! To keep these in sync, the shape of each type is defined exactly once here,
//! via the higher-order [`for_each_vm_type!`] macro, and each consumer
//! generates its view of the type from that single source of truth.
/// Invoke the given macro `$mac` once, passing it the definitions of each of the
/// `VM*` types whose layout is shared between the runtime, compilation offsets,
/// and Cranelift alias regions.
///
/// This is a higher-order macro: callers define a `macro_rules!` macro that
/// matches the grammar defined below and pass its name as an argument to this
/// macro's invocation, e.g. `for_each_vm_type!(define_vm_types)`.
///
/// # Grammar
///
/// Each type is either a `struct` or a `#[repr(C, _)]` `enum`, preceded by:
///
/// * Doc-comment attributes (`#[doc = "..."]`).
///
/// * An optional `#[cfg(...)]` attribute, gating the type on the Cargo features
/// whose code actually uses it.
///
/// Only the runtime's `struct` definition (and its layout test) is gated by
/// it. A type's offsets and alias regions are always generated because the
/// `wasmtime-environ` and `wasmtime-internal-cranelift` crates do not have
/// Cargo features for all the various Wasm features.
///
/// * An optional `#[derive(...)]` attribute.
///
/// * A `#[repr(...)]` attribute.
///
/// * A `#[snake_name = <ident>]` attribute giving the type's name in
/// `snake_case`, used to generate accessor method names.
///
/// Each field may be preceded by doc-comment attributes and, optionally, these
/// marker attributes, in this order:
///
/// * `#[aggregate]`: this field is a composite (a nested struct or array)
/// rather than a single scalar. Compiled Wasm code accesses such a field's
/// interior piecewise, so there is no one Cranelift type for the field as a
/// whole and no alias-region accessor is generated for it. The field's offset
/// is still generated, since that is what interior accesses are computed
/// relative to.
///
/// * `#[indexed]`: this field is a fixed-size array of a scalar type, whose
/// elements compiled Wasm code accesses individually by a compile-time
/// constant index. Like `#[aggregate]`, the field has no single Cranelift
/// type; unlike `#[aggregate]`, its elements all do, so an alias-region
/// accessor taking that index is generated for it and each element gets its
/// own alias region.
///
/// * `#[readonly]` and/or `#[can_move]`: describe how Cranelift may treat loads
/// and stores of that field.
///
/// ## Enums
///
/// A type may also be an `enum`, which must be `#[repr(C, $tag)]` for an
/// integer `$tag`:
///
/// ```ignore
/// #[derive(Debug)]
/// #[repr(C, u32)]
/// #[snake_name = vm_example]
/// pub enum VMExample {
/// #[snake_name = variant_a]
/// VariantA {
/// field: u32,
/// } = 0,
///
/// #[snake_name = variant_b]
/// VariantB {
/// other: u64,
/// } = 1,
/// }
/// ```
///
/// Rust lays such an enum out as a `#[repr(C)]` struct of a `$tag` tag followed
/// by a union of the variants, which is what lets the same offset machinery
/// describe it. The `C` is what guarantees that, so it is required.
///
/// Beyond a struct's rules, each variant needs an explicit discriminant (the
/// tag values compiled code compares against), braces even when empty, and its
/// own `#[snake_name]` for its generated accessors. Later attributes reach only
/// the runtime definition; `#[cfg]` is not among them, since a type's layout
/// must not vary with Cargo features. Variant fields take a struct field's
/// marker attributes but no visibility.
///
/// An enum gets two alias regions rather than a struct's one per field: one for
/// the tag, one shared by all variants' fields, because they overlap.
#[macro_export]
macro_rules! for_each_vm_type {
($mac:ident) => {
$mac! {
/// The fields compiled code needs to access to utilize a WebAssembly linear
/// memory defined within the instance, namely the start address and the
/// size in bytes.
#[derive(Debug)]
#[repr(C)]
#[snake_name = vm_memory_definition]
pub struct VMMemoryDefinition {
/// The start address.
pub base: VmPtr<u8>,
/// The current logical size of this linear memory in bytes.
///
/// This is atomic because shared memories must be able to grow their length
/// atomically. For relaxed access, see
/// [`VMMemoryDefinition::current_length()`].
pub current_length: AtomicUsize,
}
/// The fields compiled code needs to access to utilize a WebAssembly table
/// defined within the instance.
#[derive(Debug, Copy, Clone)]
#[repr(C)]
#[snake_name = vm_table_definition]
pub struct VMTableDefinition {
/// Pointer to the table data.
pub base: VmPtr<u8>,
/// The current number of elements in the table.
pub current_elements: usize,
}
/// A WebAssembly tag defined within the instance.
#[derive(Debug)]
#[repr(C)]
#[snake_name = vm_tag_definition]
pub struct VMTagDefinition {
/// Function signature's type id.
pub type_index: VMSharedTypeIndex,
}
/// The VM caller-checked "funcref" record, for caller-side signature checking.
///
/// It consists of function pointer(s), a type id to be checked by the
/// caller, and the vmctx closure associated with this function.
#[derive(Debug, Clone)]
#[repr(C)]
#[snake_name = vm_func_ref]
pub struct VMFuncRef {
/// Function pointer for this funcref if being called via the "array"
/// calling convention that `Func::new` et al use.
pub array_call: VmPtr<VMArrayCallFunction>,
/// Function pointer for this funcref if being called via the calling
/// convention we use when compiling Wasm.
///
/// Most functions come with a function pointer that we can use when they
/// are called from Wasm. The notable exception is when we `Func::wrap` a
/// host function, and we don't have a Wasm compiler on hand to compile a
/// Wasm-to-native trampoline for the function. In this case, we leave
/// `wasm_call` empty until the function is passed as an import to Wasm (or
/// otherwise exposed to Wasm via tables/globals). At this point, we look up
/// a Wasm-to-native trampoline for the function in the Wasm's compiled
/// module and use that fill in `VMFunctionImport::wasm_call`. **However**
/// there is no guarantee that the Wasm module has a trampoline for this
/// function's signature. The Wasm module only has trampolines for its
/// types, and if this function isn't of one of those types, then the Wasm
/// module will not have a trampoline for it. This is actually okay, because
/// it means that the Wasm cannot actually call this function. But it does
/// mean that this field needs to be an `Option` even though it is non-null
/// the vast vast vast majority of the time.
///
/// Once a `VMFuncRef` is exposed to compiled code this field
/// never changes again, so accesses of it are `readonly`. It is
/// not `can_move`, however: the load may be the one that traps
/// on a null funcref, and moving it would move that trap.
#[readonly]
pub wasm_call: Option<VmPtr<VMWasmCallFunction>>,
/// Function signature's type id.
///
/// See the note about `readonly` and not `can_move` on
/// `wasm_call`.
#[readonly]
pub type_index: VMSharedTypeIndex,
/// The VM state associated with this function.
///
/// The actual definition of what this pointer points to depends on the
/// function being referenced: for core Wasm functions, this is a `*mut
/// VMContext`, for host functions it is a `*mut VMHostFuncContext`, and for
/// component functions it is a `*mut VMComponentContext`.
///
/// See the note about `readonly` and not `can_move` on
/// `wasm_call`.
#[readonly]
pub vmctx: VmPtr<VMOpaqueContext>,
}
/// An imported function.
///
/// Basically the same as `VMFuncRef`, except that `wasm_call` is not optional.
#[derive(Debug, Clone)]
#[repr(C)]
#[snake_name = vm_function_import]
pub struct VMFunctionImport {
/// Same as `VMFuncRef::array_call`.
#[readonly]
#[can_move]
pub array_call: VmPtr<VMArrayCallFunction>,
/// Same as `VMFuncRef::wasm_call`, except always non-null. Must be filled
/// in by the time Wasm is importing this function!
#[readonly]
#[can_move]
pub wasm_call: VmPtr<VMWasmCallFunction>,
/// Function signature's _actual_ type id.
///
/// This is the type that the function was defined with, not the type that
/// it was imported as. These two can be different in the face of subtyping
/// and we need the former for to correctly implement dynamic downcasts.
#[readonly]
#[can_move]
pub type_index: VMSharedTypeIndex,
/// Same as `VMFuncRef::vmctx`.
#[readonly]
#[can_move]
pub vmctx: VmPtr<VMOpaqueContext>,
}
/// The fields compiled code needs to access to utilize a WebAssembly table
/// imported from another instance.
#[derive(Debug, Copy, Clone)]
#[repr(C)]
#[snake_name = vm_table_import]
pub struct VMTableImport {
/// A pointer to the imported table description.
#[readonly]
#[can_move]
pub from: VmPtr<VMTableDefinition>,
/// A pointer to the `VMContext` that owns the table description.
#[readonly]
#[can_move]
pub vmctx: VmPtr<VMContext>,
/// The table index, within `vmctx`, this definition resides at.
#[readonly]
#[can_move]
pub index: DefinedTableIndex,
}
/// The fields compiled code needs to access to utilize a WebAssembly linear
/// memory imported from another instance.
#[derive(Debug, Copy, Clone)]
#[repr(C)]
#[snake_name = vm_memory_import]
pub struct VMMemoryImport {
/// A pointer to the imported memory description.
#[readonly]
#[can_move]
pub from: VmPtr<VMMemoryDefinition>,
/// A pointer to the `VMContext` that owns the memory description.
#[readonly]
#[can_move]
pub vmctx: VmPtr<VMContext>,
/// The index of the memory in the containing `vmctx`.
#[readonly]
#[can_move]
pub index: DefinedMemoryIndex,
}
/// The fields compiled code needs to access to utilize a WebAssembly global
/// variable imported from another instance.
///
/// Note that unlike with functions, tables, and memories, `VMGlobalImport`
/// doesn't include a `vmctx` pointer. Globals are never resized, and don't
/// require a `vmctx` pointer to access.
#[derive(Debug, Copy, Clone)]
#[repr(C)]
#[snake_name = vm_global_import]
pub struct VMGlobalImport {
/// A pointer to the imported global variable description.
#[readonly]
#[can_move]
pub from: VmPtr<VMGlobalDefinition>,
/// A pointer to the context that owns the global.
///
/// Exactly what's stored here is dictated by `kind` below. This is `None`
/// for `VMGlobalKind::Host`, it's a `VMContext` for
/// `VMGlobalKind::Instance`, and it's `VMComponentContext` for
/// `VMGlobalKind::ComponentFlags`.
#[readonly]
#[can_move]
pub vmctx: Option<VmPtr<VMOpaqueContext>>,
/// The kind of global, and extra location information in addition to
/// `vmctx` above.
#[readonly]
#[can_move]
pub kind: VMGlobalKind,
}
/// The fields compiled code needs to access to utilize a WebAssembly
/// tag imported from another instance.
#[derive(Debug, Copy, Clone)]
#[repr(C)]
#[snake_name = vm_tag_import]
pub struct VMTagImport {
/// A pointer to the imported tag description.
#[readonly]
#[can_move]
pub from: VmPtr<VMTagDefinition>,
/// The instance that owns this tag.
#[readonly]
#[can_move]
pub vmctx: VmPtr<VMContext>,
/// The index of the tag in the containing `vmctx`.
#[readonly]
#[can_move]
pub index: DefinedTagIndex,
}
/// Structure that holds all mutable context that is shared across all instances
/// in a store, for example data related to fuel or epochs.
///
/// `VMStoreContext`s are one-to-one with `wasmtime::Store`s, the same way that
/// `VMContext`s are one-to-one with `wasmtime::Instance`s. And the same way
/// that multiple `wasmtime::Instance`s may be associated with the same
/// `wasmtime::Store`, multiple `VMContext`s hold a pointer to the same
/// `VMStoreContext` when they are associated with the same `wasmtime::Store`.
#[derive(Debug)]
// NB: `align(8)` is forced rather than inferred because the i386
// System V ABI aligns 64-bit integers to 4 bytes, and `VMOffsets`
// can't tell that target apart from the ones that align them to 8,
// since it only knows the target's pointer width.
#[repr(C, align(8))]
#[snake_name = vm_store_context]
pub struct VMStoreContext {
// NB: 64-bit integer fields are located first with pointer-sized fields
// trailing afterwards. That makes the offsets in this structure easier to
// calculate on 32-bit platforms as we don't have to worry about the
// alignment of 64-bit integers.
//
/// Indicator of how much fuel has been consumed and is remaining to
/// WebAssembly.
///
/// This field is typically negative and increments towards positive. Upon
/// turning positive a wasm trap will be generated. This field is only
/// modified if wasm is configured to consume fuel.
pub fuel_consumed: UnsafeCell<i64>,
/// Deadline epoch for interruption: if epoch-based interruption
/// is enabled and the global (per engine) epoch counter is
/// observed to reach or exceed this value, the guest code will
/// yield if running asynchronously.
pub epoch_deadline: UnsafeCell<u64>,
/// The "store version".
///
/// This is used to test whether stack-frame handles referring to
/// suspended stack frames remain valid.
///
/// The invariant that this upward-counting number must satisfy
/// is: the number must be incremented whenever execution starts
/// or resumes in the `Store` or when any stack is
/// dropped/freed. That way, if we take a reference to some
/// suspended stack frame and track the "version" at the time we
/// took that reference, if the version still matches, we can be
/// sure that nothing could have unwound the referenced Wasm
/// frame.
///
/// This version number is incremented in exactly one place: the
/// Wasm-to-host trampolines, after return from host code. Note
/// that this captures both the normal "return into Wasm" case
/// (where Wasm frames can subsequently return normally and thus
/// invalidate frames), and the "trap/exception unwinds Wasm
/// frames" case, which is done internally via the `raise` libcall
/// invoked after the main hostcall returns an error, and after we
/// increment this version number.
///
/// Note that this also handles the fiber/future-drop case because
/// because we *always* return into the trampoline to clean up;
/// that trampoline immediately raises an error and uses the
/// longjmp-like unwind within Cranelift frames to skip over all
/// the guest Wasm frames, but not before it increments the
/// store's execution version number.
///
/// This field is in use only if guest debugging is enabled.
pub execution_version: u64,
/// Current stack limit of the wasm module.
///
/// For more information see `crates/cranelift/src/lib.rs`.
pub stack_limit: UnsafeCell<usize>,
/// The `VMMemoryDefinition` for this store's GC heap.
#[aggregate]
pub gc_heap: UnsafeCell<VMMemoryDefinition>,
/// The value of the frame pointer register in the trampoline used
/// to call from Wasm to the host.
///
/// Maintained by our Wasm-to-host trampoline, and cleared just
/// before calling into Wasm in `catch_traps`.
///
/// This member is `0` when Wasm is actively running and has not called out
/// to the host.
///
/// Used to find the start of a contiguous sequence of Wasm frames
/// when walking the stack. Note that we record the FP of the
/// *trampoline*'s frame, not the last Wasm frame, because we need
/// to know the SP (bottom of frame) of the last Wasm frame as
/// well in case we need to resume to an exception handler in that
/// frame. The FP of the last Wasm frame can be recovered by
/// loading the saved FP value at this FP address.
pub last_wasm_exit_trampoline_fp: UnsafeCell<usize>,
/// The last Wasm program counter before we called from Wasm to the host.
///
/// Maintained by our Wasm-to-host trampoline, and cleared just before
/// calling into Wasm in `catch_traps`.
///
/// This member is `0` when Wasm is actively running and has not called out
/// to the host.
///
/// Used when walking a contiguous sequence of Wasm frames.
pub last_wasm_exit_pc: UnsafeCell<usize>,
/// The last host stack pointer before we called into Wasm from the host.
///
/// Maintained by our host-to-Wasm trampoline. This member is `0` when Wasm
/// is not running, and it's set to nonzero once a host-to-wasm trampoline
/// is executed.
///
/// When a host function is wrapped into a `wasmtime::Func`, and is then
/// called from the host, then this member is not changed meaning that the
/// previous activation in pointed to by `last_wasm_exit_trampoline_fp` is
/// still the last wasm set of frames on the stack.
///
/// This field is saved/restored during fiber suspension/resumption
/// resumption as part of `CallThreadState::swap`.
///
/// This field is used to find the end of a contiguous sequence of Wasm
/// frames when walking the stack. Additionally it's used when a trap is
/// raised as part of the set of parameters used to resume in the entry
/// trampoline's "catch" block.
pub last_wasm_entry_sp: UnsafeCell<usize>,
/// Same as `last_wasm_entry_sp`, but for the `fp` of the trampoline.
pub last_wasm_entry_fp: UnsafeCell<usize>,
/// The last trap handler from a host-to-wasm entry trampoline on the stack.
///
/// This field is configured when the host calls into wasm by the trampoline
/// itself. It stores the `pc` of an exception handler suitable to handle
/// all traps (or uncaught exceptions).
pub last_wasm_entry_trap_handler: UnsafeCell<usize>,
/// Stack information used by stack switching instructions. See documentation
/// on `VMStackChain` for details.
#[aggregate]
pub stack_chain: UnsafeCell<VMStackChain>,
/// A pointer to the embedder's `T` inside a `Store<T>`, for use with the
/// `store-data-address` unsafe intrinsic.
///
/// This pointer is fixed for the lifetime of the store, so loads
/// of it are `readonly` and `can_move`.
#[readonly]
#[can_move]
pub store_data: VmPtr<()>,
/// The range, in addresses, of the guard page that is currently in use.
///
/// This field is used when signal handlers are run to determine whether a
/// faulting address lies within the guard page of an async stack for
/// example. If this happens then the signal handler aborts with a stack
/// overflow message similar to what would happen had the stack overflow
/// happened on the main thread. This field is, by default a null..null
/// range indicating that no async guard is in use (aka no fiber). In such a
/// situation while this field is read it'll never classify a fault as an
/// guard page fault.
#[aggregate]
pub async_guard_range: Range<*mut u8>,
/// The `context.{get,set}` values for the current thread in the component
/// model. This is only used for `component-model-async` and slot[1] is only
/// used for `component-model-threading`. Despite the conditional use nature
/// this is unconditionally present as it avoids the need to make logic in
/// `VMOffsets` conditional.
///
/// This is saved/restored when threads are swapped in the component model.
///
/// NB: `UnsafeCell` because JIT code writes to the slots.
#[indexed]
pub component_context: UnsafeCell<[u32; NUM_COMPONENT_CONTEXT_SLOTS]>,
/// JIT-visible current thread for the component model's sync-to-sync
/// adapter fast path.
///
/// Like `component_context`, this is unconditionally present to keep
/// `VMOffsets` logic unconditional even though it is only used when
/// `component-model-async` is enabled.
///
/// NB: `UnsafeCell` because JIT code writes to this field.
pub current_thread: UnsafeCell<VMLazyThread>,
/// Resource lends made under unforced `VMDeferredThread`s and not
/// yet reflected in any handle's `lend_count`.
///
/// Holds `LAZY_LENDERS_CAPACITY` entries.
#[readonly]
#[can_move]
pub lazy_lenders: VmPtr<VMResourceIndex>,
/// Number of occupied slots at the front of `lazy_lenders`.
///
/// NB: `UnsafeCell` because JIT code writes to this field.
pub lazy_lenders_len: UnsafeCell<usize>,
}
/// JIT-visible representation of the store's current thread for the component
/// model, encoded as a single pointer-sized integer so that generated JIT code
/// can load, store, and compare it with a handful of instructions.
///
/// This is the inline fast-path counterpart to the host-side `CurrentThread`: a
/// fused sync-to-sync adapter records a lazy deferred thread here (a pointer to
/// a `VMDeferredThread` on its own stack frame) instead of eagerly allocating a
/// `GuestTask`/`GuestThread` in the host. Host code promotes the deferred
/// thread into a real one only when it actually needs it; see
/// `StoreOpaque::force_current_thread`.
///
/// This type is a bitpacked equivalent of the following logical `enum`:
///
/// ```ignore
/// enum VMLazyThread {
/// /// No thread.
/// None,
///
/// /// The lazy thread was promoted and materialized; get it from
/// /// `ConcurrentState::current_thread`.
/// Forced,
///
/// /// The lazy thread has not been materialized, here is a pointer to the
/// /// stack-allocated data needed to do force that promotion.
/// Deferred(*mut VMDeferredThread),
/// }
/// ```
///
/// Bitpacking details:
///
/// * `None`: `0`
///
/// * `Forced`: A non-zero value with its low-bit set.
///
/// * `Deferred`: A non-zero value with its low-bit clear.
#[derive(Debug, Copy, Clone, PartialEq, Eq)]
#[repr(transparent)]
#[snake_name = vm_lazy_thread]
pub struct VMLazyThread {
/// The bitpacked thread representation described above.
///
/// Private: use the `VMLazyThread::{none,forced,deferred}` constructors
/// and the `is_*`/`as_deferred` accessors instead of touching this
/// directly.
thread: Option<VmPtr<VMDeferredThread>>,
}
/// JIT-visible encoding of a component-model resource handle, as stored in
/// the `VMStoreContext::lazy_lenders` buffer.
///
/// This is the raw version of `TypedResourceIndex` that is exposed to Wasm
/// code; use `VMResourceIndex::{new,get}` to convert.
#[derive(Debug, Copy, Clone, PartialEq, Eq)]
#[repr(C)]
#[snake_name = vm_resource_index]
pub struct VMResourceIndex {
/// The raw `TypeResourceTableIndex`, or `u32::MAX` for a host resource.
pub(crate) ty: u32,
/// The handle's index within its table.
pub(crate) index: u32,
}
/// A resource's type and "rep", as stored in a `VMHandleTableSlot`.
#[cfg(feature = "component-model")]
#[derive(Debug, Copy, Clone, PartialEq, Eq)]
#[repr(C)]
#[snake_name = vm_typed_resource]
pub struct VMTypedResource {
/// The raw `TypeResourceTableIndex`, or `u32::MAX` for a host resource.
pub(crate) ty: u32,
/// An integer whose meaning is up to whoever defined the
/// resource.
pub(crate) rep: u32,
}
/// A call's resource-borrow scope, as stored in a
/// `VMHandleTableSlot::ResourceBorrow`.
///
/// Compiled code never reads this; it needs a defined layout only because
/// it shares a slot with `lend_count`, which compiled code does read.
#[cfg(feature = "component-model")]
#[derive(Debug, Copy, Clone, PartialEq, Eq)]
#[repr(C)]
#[snake_name = vm_scope]
pub struct VMScope {
/// The kind of scope. See the `id`/`host_id` constructors.
pub(crate) kind: u32,
/// The scope's identifier, whose meaning depends on `kind`.
pub(crate) id: u32,
}
/// One instance's view of a stream or future handle, as stored in a
/// `VMHandleTableSlot::{Stream,Future}`.
///
/// As with `VMScope`, compiled code never reads this; it needs a defined
/// layout only because it sits inside a slot.
#[cfg(feature = "component-model")]
#[derive(Debug, Copy, Clone, PartialEq, Eq)]
#[repr(C)]
#[snake_name = vm_transmit_local_state]
pub struct VMTransmitLocalState {
/// The kind of transmit state. See the `VMTransmitLocalState`
/// constructors for details.
pub(crate) kind: u32,
/// Meaningful only for the `Write` and `Read` kinds.
pub(crate) done: u32,
}
/// One slot of a component instance's `HandleTable`.
///
/// An instance keeps every kind of handle in one heterogeneous table, so a
/// slot is a tagged union over all of them, with free slots threaded into a
/// free list through `Free::next`.
///
/// The layout is guaranteed because compiled adapter code reads and writes
/// slots directly; see `transfer_borrow_to_owner`.
#[cfg(feature = "component-model")]
#[derive(Debug)]
#[repr(C, u32)]
#[snake_name = vm_handle_table_slot]
pub enum VMHandleTableSlot {
/// An unallocated slot, holding the index of the next free slot.
#[snake_name = free]
Free {
/// The next free slot's index, forming a free list.
next: u32,
} = 0,
/// An owned resource handle.
#[snake_name = resource_own]
ResourceOwn {
/// The resource's type and representation.
#[aggregate]
resource: VMTypedResource,
/// How many times this handle has been lent out as a `borrow`.
///
/// A handle with a nonzero lend count cannot be removed.
///
/// NB: must be at the same offset as `Self::ResourceBorrow::lend_count`.
lend_count: u32,
} = 1,
/// A borrowed resource handle, connected to the call scope it was
/// created within.
///
/// Dropping the borrow decrements that scope's borrow count.
#[snake_name = resource_borrow]
ResourceBorrow {
/// The resource's type and representation.
#[aggregate]
resource: VMTypedResource,
/// How many times this borrow has been lent out to other calls.
///
/// NB: must be at the same offset as `Self::ResourceOwn::lend_count`.
lend_count: u32,
/// The scope this borrow was created within.
#[aggregate]
scope: VMScope,
} = 2,
/// A host task handle.
#[snake_name = host_task]
HostTask {
/// The task's representation.
rep: u32,
} = 3,
/// A guest task handle.
#[snake_name = guest_task]
GuestTask {
/// The task's representation.
rep: u32,
} = 4,
/// A guest thread handle.
#[snake_name = guest_thread]
#[cfg_attr(
not(feature = "component-model-async"),
allow(
dead_code,
reason = "slot layout should not vary with cargo features"
)
)]
GuestThread {
/// The thread's representation.
rep: u32,
} = 5,
/// A stream handle.
#[snake_name = stream]
Stream {
/// The stream's `TypeStreamTableIndex`.
ty: u32,
/// The stream's representation.
rep: u32,
/// This instance's view of the stream's state.
#[aggregate]
state: VMTransmitLocalState,
} = 6,
/// A future handle.
#[snake_name = future]
Future {
/// The future's `TypeFutureTableIndex`.
ty: u32,
/// The future's representation.
rep: u32,
/// This instance's view of the future's state.
#[aggregate]
state: VMTransmitLocalState,
} = 7,
/// A waitable-set handle.
#[snake_name = waitable_set]
WaitableSet {
/// The waitable set's representation.
rep: u32,
} = 8,
/// An error-context handle.
#[snake_name = error_context]
ErrorContext {
/// The error context's representation.
rep: u32,
} = 9,
}
/// One `HandleTable`'s slots, as compiled code sees them.
///
/// The length lets a fused adapter's inline to-owner lend bounds check a
/// guest-supplied handle. Must be rewritten whenever the buffer grows.
#[cfg(feature = "component-model")]
#[derive(Debug, Copy, Clone)]
#[repr(C)]
#[snake_name = vm_handle_table]
pub struct VMHandleTable {
/// Pointer to the slots.
pub base: VmPtr<VMHandleTableSlot>,
/// The number of slots in `base`.
pub current_slots: usize,
}
/// A deferred component-model thread.
///
/// This is an on-stack record pushed by a fused sync-to-sync adapter's fast
/// path to defer the work that the `enter_sync_call` libcall would otherwise do
/// eagerly.
///
/// The adapter allocates one of these in its own stack frame, links the
/// previous current-thread value to it via `parent`, and finally points
/// `VMStoreContext::current_thread` at it. When host code actually needs the
/// real thread, it walks the `parent` chain to materialize thread state (see
/// `StoreOpaque::force_current_thread`).
#[derive(Debug)]
#[repr(C)]
#[snake_name = vm_deferred_thread]
pub struct VMDeferredThread {
/// The previous value of `VMStoreContext::current_thread`.
pub parent: VMLazyThread,
/// Whether the callee is async-lifted (a deferred `enter_sync_call` arg).
pub callee_async: u32,
/// The callee component instance (a deferred `enter_sync_call` argument).
pub callee_instance: u32,
/// The caller thread's `context.{get,set}` slots, saved on entry and
/// restored on the fast-path exit (or recovered while forcing).
#[indexed]
pub saved_context: [u32; NUM_COMPONENT_CONTEXT_SLOTS],
/// `VMStoreContext::lazy_lenders_len` as of this frame's push, so
/// that the lends this frame recorded are everything at or after it.
///
/// Frames nest with the call stack, so that is always a contiguous
/// suffix by the time this frame exits, and exiting an unforced frame
/// discards it by resetting `lazy_lenders_len` back to this.
pub lazy_lenders_start: usize,
}
/// This type is used to save (and subsequently restore) a subset of
/// the data in `VMStoreContext`.
///
/// See documentation of `VMStackChain` for the exact uses.
#[derive(Debug, Default, Clone)]
#[repr(C)]
#[snake_name = vm_stack_limits]
pub struct VMStackLimits {
/// Saved version of the `stack_limit` field of `VMStoreContext`.
pub stack_limit: usize,
/// Saved version of the `last_wasm_entry_fp` field of
/// `VMStoreContext`.
pub last_wasm_entry_fp: usize,
/// Saved version of the `last_wasm_entry_sp` field of
/// `VMStoreContext`.
pub last_wasm_entry_sp: usize,
/// Saved version of the `last_wasm_entry_trap_handler` field of
/// `VMStoreContext`.
pub last_wasm_entry_trap_handler: usize,
}
/// A reference to a buffer ("array") allocated on a continuation's
/// stack.
///
/// The elements are of whatever type the buffer's user expects;
/// `data` is an untyped pointer, and the runtime casts it at each
/// use site.
#[derive(Debug, Clone)]
#[repr(C)]
#[snake_name = vm_host_array]
pub struct VMHostArray {
/// Number of currently occupied slots.
pub length: u32,
/// Number of slots in the data buffer. Note that this is *not*
/// the size of the buffer in bytes!
pub capacity: u32,
/// The buffer itself, which lives on the continuation's stack
/// rather than in this object.
pub data: Option<VmPtr<u8>>,
}
/// Payload values exchanged with a continuation and the metadata
/// needed to trace GC references among them.
#[derive(Debug, Clone)]
#[repr(C)]
#[snake_name = vm_payloads]
pub struct VMPayloads {
/// The payload values themselves.
#[aggregate]
pub buffer: VMHostArray,
/// One marker byte per buffer slot, indicating whether that
/// slot contains a GC reference, or `None` when no slots do.
pub gc_ref_data: Option<VmPtr<u8>>,
}
/// AddressSanitizer bookkeeping associated with a stack.
#[derive(Debug, Clone, Default)]
#[repr(C)]
#[snake_name = vm_asan_stack_support]
pub struct VMAsanStackSupport {
/// Bottom of the usable stack range reported to ASan,
/// or `None` until ASan discovers an initial stack's
/// bounds. Technically only needed by the runtime,
/// but we are using `VmPtr` here to avoid introducing
/// macro rules for `*mut u8` just for the sake of ASan
/// builds.
pub stack_bottom: Option<VmPtr<u8>>,
/// Size of the usable stack range reported to ASan.
pub stack_size: usize,
/// The stack whose bounds ASan will report when the next
/// stack-switch handshake is completed on this stack.
/// `None` when no handshake is pending.
pub pending_source_csi: Option<VmPtr<VMCommonStackInformation>>,
}
/// The information saved for every stack, whether it is a
/// continuation's or the initial stack's.
#[derive(Debug, Clone)]
#[repr(C)]
#[snake_name = vm_common_stack_information]
pub struct VMCommonStackInformation {
/// The subset of `VMStoreContext` saved for this stack.
#[aggregate]
pub limits: VMStackLimits,
/// Where this stack is in its life cycle; a `VMStackState`
/// discriminant.
pub state: VMStackState,
/// The tags this stack handles, set while it is a `Parent`.
#[aggregate]
pub handlers: VMHostArray,
/// The index within `handlers` of the first `switch` handler.
pub first_switch_handler_index: u32,
/// AddressSanitizer bookkeeping for this stack.
#[aggregate]
pub asan: VMAsanStackSupport,
}
/// A continuation.
#[repr(C)]
#[snake_name = vm_cont_ref]
pub struct VMContRef {
/// The information saved for this continuation's stack.
#[aggregate]
pub common_stack_information: VMCommonStackInformation,
/// This continuation's parent: another continuation, the
/// initial stack, or absent.
#[aggregate]
pub parent_chain: VMStackChain,
/// Revision counter.
pub revision: usize,
/// The end of this continuation's parent chain while it is
/// `Suspended` or `Fresh`, and `None` while it is running.
pub last_ancestor: Option<VmPtr<VMContRef>>,
/// The stack this continuation runs on.
#[aggregate]
pub stack: VMContinuationStack,
/// The arguments to, and return values of, the function passed
/// to `cont.new`.
#[aggregate]
pub args: VMPayloads,
/// The payloads passed to and from this continuation once it
/// has been suspended.
#[aggregate]
pub values: VMPayloads,
/// Tells the compiler that this structure has potential
/// self-references, through `last_ancestor`.
///
/// This is a zero-sized type in final position, so it affects
/// neither this type's size nor its alignment.
#[aggregate]
pub _marker: PhantomPinned,
}
/// A slight variation of `VMContObj` which allows the
/// `contref` to be instantiated to null,
/// i.e. `Option::None`. This representation is used by
/// the GC infrastructure to construct a canonical
/// null-esque continuation object.
#[cfg(all(feature = "gc", feature = "stack-switching"))]
#[repr(C)]
#[snake_name = vm_raw_cont_obj]
pub struct VMRawContObj {
pub contref: Option<VmPtr<u8>>,
pub revision: usize,
}
/// The deferred-reference-counting collector's JIT-accessible heap
/// data.
///
/// This is a separate allocation, reached through the
/// `VMContext::gc_heap_data` pointer. Its fields are not GC heap
/// locations, so they get this type's own alias regions rather than
/// the GC heap's.
///
/// `wasmtime::runtime::vm::gc::enabled::drc` owns one of these,
/// wrapped in a cell because compiled Wasm writes to it, and
/// accesses it only through that wrapper's methods.
#[cfg(feature = "gc-drc")]
#[derive(Default)]
#[repr(C)]
#[snake_name = vm_drc_heap_data]
pub struct VMDrcHeapData {
/// The head of the over-approximated-stack-roots list.
pub over_approximated_stack_roots: Option<VMGcRef>,
/// The current size of the over-approximated-stack-roots list.
pub current_over_approximated_stack_roots_len: u32,
/// The size of the over-approximated-stack-roots list
/// immediately after the last GC.
pub over_approximated_stack_roots_len_after_last_gc: u32,
}
/// The copying collector's JIT-accessible bump-allocation state.
///
/// Like [`VMDrcHeapData`], this is a separate allocation reached
/// through `VMContext::gc_heap_data`, and is owned, wrapped in a
/// cell, by `wasmtime::runtime::vm::gc::enabled::copying`.
#[cfg(feature = "gc-copying")]
#[derive(Default)]
#[repr(C)]
#[snake_name = vm_copying_heap_data]
pub struct VMCopyingHeapData {
/// Current bump pointer (an index into the GC heap).
pub bump_ptr: u32,
/// End of the active semi-space.
pub active_space_end: u32,
}
/// The null collector's JIT-accessible bump-allocation state.
///
/// Unlike the two above, this is not a separate allocation: it is
/// the first field of `NullHeap` (in
/// `wasmtime::runtime::vm::gc::enabled::null`), again wrapped in a
/// cell, and compiled Wasm reaches it through a pointer to that
/// field.
#[cfg(feature = "gc-null")]
#[repr(C)]
#[snake_name = vm_null_heap_data]
pub struct VMNullHeapData {
/// The bump-allocation finger, an index into the GC heap.
pub next: NonZeroU32,
}
/// The common header for all objects allocated in a GC heap.
///
/// This header is shared across all collectors, although particular
/// collectors may always add their own trailing fields to this
/// header for all of their own GC objects.
///
/// This is a bit-packed structure that logically has the following
/// fields:
///
/// ```ignore
/// struct VMGcHeader {
/// // Highest 5 bits.
/// kind: VMGcKind,
///
/// // 27 bits available for the `GcRuntime` to make use of
/// // however it sees fit.
/// reserved: u27,
///
/// // The `VMSharedTypeIndex` for this GC object, if it isn't an
/// // `externref` (or an `externref` re-wrapped as an
/// // `anyref`). `None` is represented with
/// // `VMSharedTypeIndex::reserved_value()`.
/// ty: Option<VMSharedTypeIndex>,
/// }
/// ```
///
/// NB: the `kind` and `reserved` fields share one word, and
/// therefore one alias region: splitting them would let Cranelift
/// forward a stale kind across a reserved-bits store.
#[derive(Debug, Clone, Copy)]
#[repr(C, align(8))]
#[snake_name = vm_gc_header]
pub struct VMGcHeader {
/// The object's `VMGcKind` and 27 bits of space reserved for
/// however the GC sees fit to use it.
pub(crate) kind: u32,
/// The object's type index.
pub(crate) ty: VMSharedTypeIndex,
}
/// The common header for all objects in the DRC collector.
///
/// This adds a ref count and over-approximated-stack-roots list
/// link on top of the collector-agnostic [`VMGcHeader`].
#[cfg(feature = "gc-drc")]
#[repr(C)]
#[snake_name = vm_drc_header]
pub(crate) struct VMDrcHeader {
/// The collector-agnostic header.
#[aggregate]
pub(crate) header: VMGcHeader,
/// This object's reference count.
pub(crate) ref_count: u64,
/// The next object in the over-approximated-stack-roots list,
/// if this object is in that list.
pub(crate) next_over_approximated_stack_root: Option<VMGcRef>,
/// The size of this object in the GC heap.
///
/// Written by the runtime at allocation time; compiled Wasm
/// never accesses it.
pub(crate) object_size: u32,
}
/// The common header for all objects in the copying collector.
#[cfg(feature = "gc-copying")]
#[repr(C)]
#[snake_name = vm_copying_header]
pub(crate) struct VMCopyingHeader {
/// The collector-agnostic header.
#[aggregate]
pub(crate) header: VMGcHeader,
/// The size of this object in the GC heap.
pub(crate) object_size: u32,
}
}
};
}