API reference
python:aix:aix.lexer.core.TokenList
class TokenList
TokenList(tokens: list[Token])
Source: packages/aix/src/aix/lexer/core.py:177
Simple token stream consumed by the parser.
Attributes
-
tokens — Type: list[Token]
-
position — Type: int
-
cur_tok — Type: Token
Methods
-
__init__ — Initialize the token stream.
-
__iter__ — Reset iteration and return this stream.
-
__next__ — Return the next token for iteration.
-
get_next_token — Advance parser cursor and return the current token.
-
get_token — Return the next token and advance.