Skip to content

armgit / moli

Public Imported from GitHub
0

Best headless browser for AI agents. Lite, Fast, High-Compatibility. Built in Rust

1.0 KB
Raw Blame
//! Shared tokenization primitives for structured HTTP header field values.
//!
//! The observable behavior follows Blink's `HeaderFieldTokenizer` normal and
//! relaxed modes. Higher-level crates remain responsible for deciding which
//! tokens, separators, and parameters are valid for a particular header.
//!
//! One compatibility detail is intentionally non-standard: Blink currently
//! accepts DEL (`0x7f`) as a token character even though MIME token grammar
//! classifies it as a control. This crate preserves that behavior. Relaxed mode
//! also admits MIME `tspecials` other than space, semicolon, and quote, matching
//! Blink rather than defining another standards-compliance mode.

mod fields;
mod parameters;
mod tokenizer;

pub use fields::{
    HeaderFields, decode_header_value, deserialize_headers, header_value_from_byte_string,
    headers_from_byte_strings, headers_to_byte_strings,
};
pub use parameters::{split_outside_quoted_strings, unquote_parameter_value};
pub use tokenizer::{HeaderFieldTokenMode, HeaderFieldTokenizer};

#[cfg(test)]
mod tests;