API reference
python:arx:arx.lexer.Lexer
class Lexer
Lexer()
Source: packages/arx/src/arx/lexer/core.py:345
Declared as: python:arx:arx.lexer.core.Lexer
Lexer class for tokenizing known variables.
Attributes
-
lex_loc — Source location for lexer.; Type: SourceLocation
-
last_char — Last char.; Type: str
-
initialized — Whether to use initialized.; Type: bool
-
new_line — Whether to use new line.; Type: bool
-
_keyword_map — Keyword map.; Type: dict[str, TokenKind]
-
_line_comment_delims — Line comment delims.; Type: tuple[str, ...]
-
_multi_char_ops — Multi char ops.; Type: set[str]
-
_literal_keywords — Literal keywords.; Type: dict[str, Any]
-
_keyword_token_map — Keyword token map.; Type: dict[str, TokenKind]
Methods
-
__init__ — Initialize Lexer.
-
_parse_docstring — Parse docstrings delimited by triple backticks.
-
_parse_operator — Parse multi-character operators.
-
_parse_quoted_literal — Parse quoted string or character literals.
-
advance — Advance the token from the buffer.
-
clean — Reset the Lexer attributes.
-
get_token — Get the next token.
-
lex — Create a list of tokens from input source.