API reference
python:arx:arx.lexer.core.Lexer
class Lexer
Lexer()
Source: packages/arx/src/arx/lexer/core.py:352
Lexer class for tokenizing known variables.
Attributes
-
lex_loc — Type: SourceLocation; Source location for lexer.
-
last_char — Type: str
-
initialized — Type: bool
-
new_line — Type: bool
-
_keyword_map — Type: dict[str, TokenKind]
-
_line_comment_delims — Type: tuple[str, Ellipsis]
-
_multi_char_ops — Type: set[str]
-
_literal_keywords — Type: dict[str, Any]
-
_keyword_token_map — Type: dict[str, TokenKind]
Methods
-
__init__ — Initialize Lexer.
-
_parse_docstring — Parse docstrings delimited by triple backticks.
-
_parse_operator — Parse multi-character operators.
-
_parse_quoted_literal — Parse quoted string or character literals.
-
advance — Advance the token from the buffer.
-
clean — Reset the Lexer attributes.
-
get_token — Get the next token.
-
lex — Create a list of tokens from input source.