G3D::TextInput::Settings Class Reference

Tokenizer configuration options. More...

Public Member Functions
	Settings ()

Public Attributes
bool	caseSensitive
	When parsing booleans and msvcFloatSpecials, is case significant? Default is {true}.
bool	cppBlockComments
	If true, C-style slash-star marks a multi-line comment.
bool	cppLineComments
	If true, // begins a single line comment.
bool	escapeSequencesInStrings
	If true, \r, \n, \t, \0, \\ and other escape sequences inside strings are converted to the equivalent C++ escaped character.
Set< std::string >	falseSymbols
	See trueSymbols.
bool	generateCommentTokens
	If true, comments enabled by cppBlockComments, cppLineComments and otherLineComments will generate their respective tokens.
bool	generateNewlineTokens
	If true, newlines will generate tokens.
bool	msvcFloatSpecials
	Parse -1.
char	otherCommentCharacter
	If not '\0', specifies a character that begins single line comments ('#' and '' are popular choices).
char	otherCommentCharacter2
	Another (optional) 1-comment character.
bool	otherLineComments
	If true, otherCommentCharacter and otherCommentCharacter2 are used to begin single line comments in the same way cppLineComments is.
bool	proofSymbols
	Parse the following set of useful proof symbols:
bool	signedNumbers
	If true, "-1" parses as the number -1 instead of the symbol "-" followed by the number 1.
bool	simpleFloatSpecials
	Parses "+inf', "-inf", "inf", "nan" as floats instead of symbols.
char	singleQuoteCharacter
	The character to use as a single quote.
bool	singleQuotedStrings
	If true, strings can be marked with single quotes (e.g., 'aaa').
std::string	sourceFileName
	If set to a non-empty string, that string will be used in place of the real file name (or in place of a pseudonym constructed from the buffer if given FROM_STRING) in tokens and exceptions.
int	startingLineNumberOffset
	Added to the line number reported by peekLineNumber and in exceptions.
Set< std::string >	trueSymbols
	All symbols that will become the 'true' boolean token.

Detailed Description

Tokenizer configuration options.

Constructor & Destructor Documentation

G3D::TextInput::Settings::Settings

(

)

Member Data Documentation

bool G3D::TextInput::Settings::caseSensitive

When parsing booleans and msvcFloatSpecials, is case significant? Default is {true}.

bool G3D::TextInput::Settings::cppBlockComments

If true, C-style slash-star marks a multi-line comment.

See generateCommentTokens for rules on how this is applied.

Default is true.

bool G3D::TextInput::Settings::cppLineComments

If true, // begins a single line comment.

See generateCommentTokens for rules on how this is applied.

Default is true.

bool G3D::TextInput::Settings::escapeSequencesInStrings

If true, \r, \n, \t, \0, \\ and other escape sequences inside strings are converted to the equivalent C++ escaped character.

If false, backslashes are treated literally. It is convenient to set to false if reading Windows paths, for example, like c:\foo\bar.

Default is true.

Set<std::string> G3D::TextInput::Settings::falseSymbols

See trueSymbols.

Default is {false}

bool G3D::TextInput::Settings::generateCommentTokens

If true, comments enabled by cppBlockComments, cppLineComments and otherLineComments will generate their respective tokens.

If false, the same settings will enable parsing and ignoring comments

Default is false.

bool G3D::TextInput::Settings::generateNewlineTokens

If true, newlines will generate tokens.

If false, newlines will be discarded as whitespace when parsed outside of other tokens.

Default is false.

bool G3D::TextInput::Settings::msvcFloatSpecials

Parse -1.

#IND00 as the floating point number returned by nan(), -1.#INF00 as -G3Dinf(), and 1.#INF00 as G3D::inf().

Note that the C99 standard specifies that a variety of formats like "nan" are to be used; these are supported by G3D::TextInput::Settings::simpleFloatSpecials.

An alternative to specifying msvcFloatSpecials is to read numbers as:

            Token x = t.read();
            Token y = t.peek();
            if ((x.string() == "-1.") && 
                (y.string() == "#INF00") && 
                (y.character() == x.character() + 3) &&
                (y.line() == x.line()) {
                t.read();
                return nan();
            }
... similar cases for inf
          

If the single-comment character was #, the floating point special format overrides the comment and will be parsed instead.

If signedNumbers is false msvcFloatSpecials will not be parsed.

Default is true.

char G3D::TextInput::Settings::otherCommentCharacter

If not '\0', specifies a character that begins single line comments ('#' and '' are popular choices).

This is independent of the cppLineComments flag. If the character appears in text with a backslash in front of it, it is considered escaped and is not treated as a comment character.

Default is '\0'.

char G3D::TextInput::Settings::otherCommentCharacter2

Another (optional) 1-comment character.

Useful for files that support multiple comment syntaxes. Default is '\0'.

bool G3D::TextInput::Settings::otherLineComments

If true, otherCommentCharacter and otherCommentCharacter2 are used to begin single line comments in the same way cppLineComments is.

See generateCommentTokens for rules on how this is applied.

Default is true.

bool G3D::TextInput::Settings::proofSymbols

Parse the following set of useful proof symbols:

=> ::> <:: :> <: |- ::= := <-

Default is false.

bool G3D::TextInput::Settings::signedNumbers

If true, "-1" parses as the number -1 instead of the symbol "-" followed by the number 1.

Default is true.

bool G3D::TextInput::Settings::simpleFloatSpecials

Parses "+inf', "-inf", "inf", "nan" as floats instead of symbols.

Defaults to true.

char G3D::TextInput::Settings::singleQuoteCharacter

The character to use as a single quote.

Defaults to "'" (backquote), occasionally useful to set to "`" (forward quote) or to "," (comma) for reading CSV files.

bool G3D::TextInput::Settings::singleQuotedStrings

If true, strings can be marked with single quotes (e.g., 'aaa').

If false, the quote character is parsed as a symbol. Default is true. Backquote (`) is always parsed as a symbol.

std::string G3D::TextInput::Settings::sourceFileName

If set to a non-empty string, that string will be used in place of the real file name (or in place of a pseudonym constructed from the buffer if given FROM_STRING) in tokens and exceptions.

Default is empty.

int G3D::TextInput::Settings::startingLineNumberOffset

Added to the line number reported by peekLineNumber and in exceptions.

Useful for concatenating files that are parsed separately. Default is zero.

Set<std::string> G3D::TextInput::Settings::trueSymbols

All symbols that will become the 'true' boolean token.

See also caseSensitive. Clear this value to disable parsing of true booleans.

Default is {true}.