Enum Token

pub enum Token<'input> {
Show 223 variants    Add,
    Addrspace,
    Adv,
    AdvMap,
    InsertHdword,
    InsertHdwordWithDomain,
    InsertHqword,
    InsertHperm,
    InsertMem,
    AdvLoadw,
    AdvPipe,
    AdvPush,
    AdvStack,
    PushMapval,
    PushMapvalCount,
    PushMapvaln,
    PushMtnode,
    And,
    Assert,
    Assertz,
    AssertEq,
    AssertEqw,
    EvalCircuit,
    Begin,
    Byte,
    Caller,
    Call,
    Cdrop,
    Cdropw,
    Clk,
    Const,
    CryptoStream,
    Cswap,
    Cswapw,
    Debug,
    Div,
    Drop,
    Dropw,
    Dup,
    Dupw,
    Dynexec,
    Dyncall,
    Else,
    Emit,
    End,
    Enum,
    Eq,
    Eqw,
    Ext2Add,
    Ext2Div,
    Ext2Inv,
    Ext2Mul,
    Ext2Neg,
    Ext2Sub,
    Err,
    Exec,
    Export,
    Exp,
    ExpU,
    False,
    Felt,
    FriExt2Fold4,
    Gt,
    Gte,
    Hash,
    HasMapkey,
    HornerBase,
    HornerExt,
    LogPrecompile,
    Hperm,
    Hmerge,
    I1,
    I8,
    I16,
    I32,
    I64,
    I128,
    If,
    ILog2,
    Inv,
    IsOdd,
    Local,
    Locaddr,
    LocLoad,
    LocLoadw,
    LocLoadwBe,
    LocLoadwLe,
    LocStore,
    LocStorew,
    LocStorewBe,
    LocStorewLe,
    Lt,
    Lte,
    Mem,
    MemLoad,
    MemLoadw,
    MemLoadwBe,
    MemLoadwLe,
    MemStore,
    MemStorew,
    MemStorewBe,
    MemStorewLe,
    MemStream,
    Movdn,
    Movdnw,
    Movup,
    Movupw,
    MtreeGet,
    MtreeMerge,
    MtreeSet,
    MtreeVerify,
    Mul,
    Neg,
    Neq,
    Not,
    Nop,
    Or,
    Padw,
    Pow2,
    Proc,
    Procref,
    Ptr,
    Pub,
    Push,
    Repeat,
    Reversew,
    Reversedw,
    Range,
    Sdepth,
    Stack,
    Struct,
    Sub,
    Swap,
    Swapw,
    Swapdw,
    Syscall,
    Trace,
    True,
    Type,
    Use,
    U8,
    U16,
    U32,
    U32And,
    U32Assert,
    U32Assert2,
    U32Assertw,
    U32Cast,
    U32Div,
    U32Divmod,
    U32Gt,
    U32Gte,
    U32Lt,
    U32Lte,
    U32Max,
    U32Min,
    U32Mod,
    U32Not,
    U32Or,
    U32OverflowingAdd,
    U32OverflowingAdd3,
    U32WideningAdd,
    U32WideningAdd3,
    U32WideningMadd,
    U32WideningMul,
    U32OverflowingSub,
    U32Popcnt,
    U32Clz,
    U32Ctz,
    U32Clo,
    U32Cto,
    U32Rotl,
    U32Rotr,
    U32Shl,
    U32Shr,
    U32Split,
    U32Test,
    U32Testw,
    U32WrappingAdd,
    U32WrappingAdd3,
    U32WrappingMadd,
    U32WrappingMul,
    U32WrappingSub,
    U32Xor,
    U64,
    U128,
    While,
    Word,
    Event,
    Xor,
    At,
    Bang,
    Colon,
    ColonColon,
    Dot,
    Comma,
    Equal,
    Langle,
    Lparen,
    Lbrace,
    Lbracket,
    Minus,
    Plus,
    Semicolon,
    SlashSlash,
    Slash,
    Star,
    Rangle,
    Rparen,
    Rbrace,
    Rbracket,
    Rstab,
    DocComment(DocumentationType),
    HexValue(IntValue),
    HexWord(WordValue),
    BinValue(BinEncodedValue),
    Int(u64),
    Ident(&'input str),
    ConstantIdent(&'input str),
    QuotedIdent(&'input str),
    QuotedString(&'input str),
    Comment,
    Eof,
}

Expand description

The token type produced by crate::parser::Lexer, and consumed by the parser.

Variants§

§

Eof

Implementations§

§

impl<'input> Token<'input>

pub fn is_instruction(&self) -> bool

Returns true if this token represents the name of an instruction.

This is used to simplify diagnostic output related to expected tokens so as not to overwhelm the user with a ton of possible expected instruction variants.

pub fn is_type_keyword(&self) -> bool

Returns true if this token represents the name of an type or a type-related keyword.

This is used to simplify diagnostic output related to expected tokens so as not to overwhelm the user with a ton of possible expected tokens.

pub fn keyword_searcher() -> AhoCorasick

Constructs a DFA capable of recognizing Miden Assembly keywords.

Constructing the state machine is expensive, so it should not be done in hot code. Instead, prefer to construct it once and reuse it many times.

Currently we construct an instance of this searcher in the lexer, which is then used to select a keyword token or construct an identifier token depending on whether a given string is a known keyword.

pub fn from_keyword_or_ident(s: &'input str) -> Token<'input>

Returns an appropriate Token depending on whether the given string is a keyword or an identifier.

NOTE: This constructs and throws away an expensive-to-construct Aho-Corasick state machine. You should not call this from any code on a hot path. Instead, construct the state machine once using Token::keyword_searcher, and reuse it for all searches using Token::from_keyword_or_ident_with_searcher.

Currently, this function is only called along one code path, which is when we are constructing a parser error in which we wish to determine which, if any, of the expected tokens are instruction opcode keywords, so we can collapse them into a more user-friendly error message. This is not on a hot path, so we don’t care if it is a bit slow.

pub fn from_keyword_or_ident_with_searcher( s: &'input str, searcher: &AhoCorasick, ) -> Token<'input>

This is the primary function you should use when you wish to get an appropriate token for a given input string, depending on whether it is a keyword or an identifier.

See Token::keyword_searcher for additional information on how this is meant to be used.

pub fn parse(s: &'input str) -> Option<Token<'input>>

Parses a Token from a string corresponding to that token.

This solely exists to aid in constructing more user-friendly error messages in certain scenarios, and is otherwise not used (nor should it be). It is quite expensive to call due to invoking Token::keyword_searcher under the covers. See the documentation for that function for more details.

Trait Implementations§

§

impl<'input> Clone for Token<'input>

§

fn clone(&self) -> Token<'input>

Returns a duplicate of the value. Read more

1.0.0 · Source§

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source. Read more

§

impl<'input> Debug for Token<'input>

§

fn fmt(&self, f: &mut Formatter<'_>) -> Result<(), Error>

Formats the value using the given formatter. Read more

§

impl Display for Token<'_>

§

fn fmt(&self, f: &mut Formatter<'_>) -> Result<(), Error>

Formats the value using the given formatter. Read more

Auto Trait Implementations§

§

impl<'input> Freeze for Token<'input>

§

impl<'input> RefUnwindSafe for Token<'input>

§

impl<'input> Send for Token<'input>

§

impl<'input> Sync for Token<'input>

§

impl<'input> Unpin for Token<'input>

§

impl<'input> UnsafeUnpin for Token<'input>

§

impl<'input> UnwindSafe for Token<'input>

Blanket Implementations§

Source §

impl<T> Any for T
where T: 'static + ?Sized,

Source §

fn type_id(&self) -> TypeId

Gets the TypeId of self. Read more

Source §

impl<T> Borrow<T> for T
where T: ?Sized,

Source §

fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more

Source §

impl<T> BorrowMut<T> for T
where T: ?Sized,

Source §

fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more

Source §

impl<T> CloneToUninit for T
where T: Clone,

Source §

unsafe fn clone_to_uninit(&self, dest: *mut u8)

🔬This is a nightly-only experimental API. (clone_to_uninit)

Performs copy-assignment from self to dest. Read more

Source §

impl<T> From<T> for T

Source §

fn from(t: T) -> T

Returns the argument unchanged.

§

impl<T> Instrument for T

§

fn instrument(self, span: Span) -> Instrumented<Self>

Instruments this type with the provided [Span], returning an Instrumented wrapper. Read more

§

fn in_current_span(self) -> Instrumented<Self>

Instruments this type with the current Span, returning an Instrumented wrapper. Read more

Source §

impl<T, U> Into for T
where U: From<T>,

Source §

fn into(self) -> U

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

Source §

impl<T> IntoEither for T

Source §

fn into_either(self, into_left: bool) -> Either<Self, Self>

Converts self into a Left variant of Either<Self, Self> if into_left is true. Converts self into a Right variant of Either<Self, Self> otherwise. Read more

Source §

fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
where F: FnOnce(&Self) -> bool,

Converts self into a Left variant of Either<Self, Self> if into_left(&self) returns true. Converts self into a Right variant of Either<Self, Self> otherwise. Read more

§

impl<D> OwoColorize for D

§

fn fg<C>(&self) -> FgColorDisplay<'_, C, Self>
where C: Color,

Set the foreground color generically Read more

§

fn bg<C>(&self) -> BgColorDisplay<'_, C, Self>
where C: Color,

Set the background color generically. Read more

§

fn black(&self) -> FgColorDisplay<'_, Black, Self>

Change the foreground color to black

§

fn on_black(&self) -> BgColorDisplay<'_, Black, Self>

Change the background color to black

§

fn red(&self) -> FgColorDisplay<'_, Red, Self>

Change the foreground color to red

§

fn on_red(&self) -> BgColorDisplay<'_, Red, Self>

Change the background color to red

§

fn green(&self) -> FgColorDisplay<'_, Green, Self>

Change the foreground color to green

§

fn on_green(&self) -> BgColorDisplay<'_, Green, Self>

Change the background color to green

§

fn yellow(&self) -> FgColorDisplay<'_, Yellow, Self>

Change the foreground color to yellow

§

fn on_yellow(&self) -> BgColorDisplay<'_, Yellow, Self>

Change the background color to yellow

§

fn blue(&self) -> FgColorDisplay<'_, Blue, Self>

Change the foreground color to blue

§

fn on_blue(&self) -> BgColorDisplay<'_, Blue, Self>

Change the background color to blue

§

fn magenta(&self) -> FgColorDisplay<'_, Magenta, Self>

Change the foreground color to magenta

§

fn on_magenta(&self) -> BgColorDisplay<'_, Magenta, Self>

Change the background color to magenta

§

fn purple(&self) -> FgColorDisplay<'_, Magenta, Self>

Change the foreground color to purple

§

fn on_purple(&self) -> BgColorDisplay<'_, Magenta, Self>

Change the background color to purple

§

fn cyan(&self) -> FgColorDisplay<'_, Cyan, Self>

Change the foreground color to cyan

§

fn on_cyan(&self) -> BgColorDisplay<'_, Cyan, Self>

Change the background color to cyan

§

fn white(&self) -> FgColorDisplay<'_, White, Self>

Change the foreground color to white

§

fn on_white(&self) -> BgColorDisplay<'_, White, Self>

Change the background color to white

§

fn default_color(&self) -> FgColorDisplay<'_, Default, Self>

Change the foreground color to the terminal default

§

fn on_default_color(&self) -> BgColorDisplay<'_, Default, Self>

Change the background color to the terminal default

§

fn bright_black(&self) -> FgColorDisplay<'_, BrightBlack, Self>

Change the foreground color to bright black

§

fn on_bright_black(&self) -> BgColorDisplay<'_, BrightBlack, Self>

Change the background color to bright black

§

fn bright_red(&self) -> FgColorDisplay<'_, BrightRed, Self>

Change the foreground color to bright red

§

fn on_bright_red(&self) -> BgColorDisplay<'_, BrightRed, Self>

Change the background color to bright red

§

fn bright_green(&self) -> FgColorDisplay<'_, BrightGreen, Self>

Change the foreground color to bright green

§

fn on_bright_green(&self) -> BgColorDisplay<'_, BrightGreen, Self>

Change the background color to bright green

§

fn bright_yellow(&self) -> FgColorDisplay<'_, BrightYellow, Self>

Change the foreground color to bright yellow

§

fn on_bright_yellow(&self) -> BgColorDisplay<'_, BrightYellow, Self>

Change the background color to bright yellow

§

fn bright_blue(&self) -> FgColorDisplay<'_, BrightBlue, Self>

Change the foreground color to bright blue

§

fn on_bright_blue(&self) -> BgColorDisplay<'_, BrightBlue, Self>

Change the background color to bright blue

§

fn bright_magenta(&self) -> FgColorDisplay<'_, BrightMagenta, Self>

Change the foreground color to bright magenta

§

fn on_bright_magenta(&self) -> BgColorDisplay<'_, BrightMagenta, Self>

Change the background color to bright magenta

§

fn bright_purple(&self) -> FgColorDisplay<'_, BrightMagenta, Self>

Change the foreground color to bright purple

§

fn on_bright_purple(&self) -> BgColorDisplay<'_, BrightMagenta, Self>

Change the background color to bright purple

§

fn bright_cyan(&self) -> FgColorDisplay<'_, BrightCyan, Self>

Change the foreground color to bright cyan

§

fn on_bright_cyan(&self) -> BgColorDisplay<'_, BrightCyan, Self>

Change the background color to bright cyan

§

fn bright_white(&self) -> FgColorDisplay<'_, BrightWhite, Self>

Change the foreground color to bright white

§

fn on_bright_white(&self) -> BgColorDisplay<'_, BrightWhite, Self>

Change the background color to bright white

§

fn bold(&self) -> BoldDisplay<'_, Self>

Make the text bold

§

fn dimmed(&self) -> DimDisplay<'_, Self>

Make the text dim

§

fn italic(&self) -> ItalicDisplay<'_, Self>

Make the text italicized

§

fn underline(&self) -> UnderlineDisplay<'_, Self>

Make the text underlined

§

fn blink(&self) -> BlinkDisplay<'_, Self>

Make the text blink

§

fn blink_fast(&self) -> BlinkFastDisplay<'_, Self>

Make the text blink (but fast!)

§

fn reversed(&self) -> ReversedDisplay<'_, Self>

Swap the foreground and background colors

§

fn hidden(&self) -> HiddenDisplay<'_, Self>

Hide the text

§

fn strikethrough(&self) -> StrikeThroughDisplay<'_, Self>

Cross out the text

§

fn color<Color>(&self, color: Color) -> FgDynColorDisplay<'_, Color, Self>
where Color: DynColor,

Set the foreground color at runtime. Only use if you do not know which color will be used at compile-time. If the color is constant, use either [OwoColorize::fg] or a color-specific method, such as [OwoColorize::green], Read more

§

fn on_color<Color>(&self, color: Color) -> BgDynColorDisplay<'_, Color, Self>
where Color: DynColor,

Set the background color at runtime. Only use if you do not know what color to use at compile-time. If the color is constant, use either [OwoColorize::bg] or a color-specific method, such as [OwoColorize::on_yellow], Read more

§