Farkle

DefaultTokenizer Type

Namespace: Farkle.Parser
Assembly: Farkle.dll
Base Type: Tokenizer

Farkle's default tokenizer, powered by a DFA.

Custom tokenizers are recommended to inherit this class to still have access to Farkle's tokenizer through the base GetNextToken method.

Constructors

Constructor Description

DefaultTokenizer(grammar)

Full Usage: DefaultTokenizer(grammar)

Parameters:
Returns: DefaultTokenizer
grammar : Grammar
Returns: DefaultTokenizer