API reference
python:arx:arx.lexer.core.TokenList
class TokenList
TokenList(tokens: list[Token])
Source: packages/arx/src/arx/lexer/core.py:246
Class for handle a List of tokens.
Attributes
-
tokens — Tokens in source order.; Type: list[Token]
-
position — Position.; Type: int
-
cur_tok — Cur tok.; Type: Token
-
ignore_indentation — Whether a parser scope ignores indentation markers.; Type: bool
Methods
-
__init__ — Instantiate a TokenList object.
-
__iter__ — Overload the iterator operation.
-
__next__ — Overload the next method used by the iteration.
-
get_next_token — Provide a simple token buffer.
-
get_token — Get the next token.