ANTLR Support Libraries 2.7.1+
Classes | Public Member Functions | Static Public Attributes | Protected Types | Protected Member Functions | Protected Attributes | Private Member Functions | Static Private Attributes

antlr::CharScanner Class Reference

#include <CharScanner.hpp>

Inheritance diagram for antlr::CharScanner:
Inheritance graph
[legend]
Collaboration diagram for antlr::CharScanner:
Collaboration graph
[legend]

List of all members.

Classes

class  Tracer

Public Member Functions

 CharScanner (InputBuffer &cb, bool case_sensitive)
 CharScanner (InputBuffer *cb, bool case_sensitive)
 CharScanner (const LexerSharedInputState &state, bool case_sensitive)
virtual ~CharScanner ()
virtual int LA (unsigned int i)
virtual void append (char c)
virtual void append (const std::string &s)
virtual void commit ()
virtual void recover (const RecognitionException &ex, const BitSet &tokenSet)
virtual void consume ()
virtual void consumeUntil (int c)
virtual void consumeUntil (const BitSet &set)
virtual unsigned int mark ()
 Mark the current position and return a id for it.
virtual void rewind (unsigned int pos)
 Rewind the scanner to a previously marked position.
virtual void match (int c)
 See if input contains character 'c' throw MismatchedCharException if not.
virtual void match (const BitSet &b)
virtual void match (const char *s)
virtual void match (const std::string &s)
virtual void matchNot (int c)
virtual void matchRange (int c1, int c2)
virtual bool getCaseSensitive () const
virtual void setCaseSensitive (bool t)
virtual bool getCaseSensitiveLiterals () const =0
virtual int getLine () const
 Get the line the scanner currently is in (starts at 1)
virtual void setLine (int l)
 set the line number
virtual int getColumn () const
 Get the column the scanner currently is in (starts at 1)
virtual void setColumn (int c)
 set the column number
virtual const std::string & getFilename () const
 get the filename for the file currently used
virtual void setFilename (const std::string &f)
 Set the filename the scanner is using (used in error messages)
virtual bool getCommitToPath () const
virtual void setCommitToPath (bool commit)
virtual const std::string & getText () const
virtual void setText (const std::string &s)
virtual void resetText ()
virtual RefToken getTokenObject () const
virtual void newline ()
virtual void tab ()
int setTabsize (int size)
 set the tabsize. Returns the old tabsize
int getTabSize () const
 Return the tabsize used by the scanner.
virtual void reportError (const RecognitionException &e)
virtual void reportError (const std::string &s)
virtual void reportWarning (const std::string &s)
virtual InputBuffergetInputBuffer ()
virtual LexerSharedInputState getInputState ()
virtual void setInputState (LexerSharedInputState state)
virtual void setTokenObjectFactory (factory_type factory)
 Set the factory for created tokens.
virtual int testLiteralsTable (int ttype) const
virtual int testLiteralsTable (const std::string &txt, int ttype) const
virtual int toLower (int c) const
 Override this method to get more specific case handling.
virtual void uponEOF ()
virtual void traceIndent ()
 Methods used to change tracing behavior.
virtual void traceIn (const char *rname)
virtual void traceOut (const char *rname)

Static Public Attributes

static const int EOF_CHAR = EOF

Protected Types

typedef RefToken(* factory_type )()

Protected Member Functions

virtual RefToken makeToken (int t)
 Create a new RefToken of type t.

Protected Attributes

std::string text
bool saveConsumedInput
 flag indicating wether consume saves characters
factory_type tokenFactory
 Factory for tokens.
bool caseSensitive
 Is this lexer case sensitive.
std::map< std::string, int,
CharScannerLiteralsLess
literals
RefToken _returnToken
 used to return tokens w/o using return val
LexerSharedInputState inputState
 Input state, gives access to input stream, shared among different lexers.
bool commitToPath
int tabsize
 tab size the scanner uses.
int traceDepth

Private Member Functions

 CharScanner (const CharScanner &other)
CharScanneroperator= (const CharScanner &other)

Static Private Attributes

static const int NO_CHAR = 0

Detailed Description

Superclass of generated lexers


Member Typedef Documentation

typedef RefToken(* antlr::CharScanner::factory_type)() [protected]

Constructor & Destructor Documentation

CharScanner::CharScanner ( InputBuffer cb,
bool  case_sensitive 
)
CharScanner::CharScanner ( InputBuffer cb,
bool  case_sensitive 
)
CharScanner::CharScanner ( const LexerSharedInputState state,
bool  case_sensitive 
)
virtual antlr::CharScanner::~CharScanner ( ) [inline, virtual]
antlr::CharScanner::CharScanner ( const CharScanner other) [private]

Member Function Documentation

virtual void antlr::CharScanner::append ( char  c) [inline, virtual]
virtual void antlr::CharScanner::append ( const std::string &  s) [inline, virtual]
virtual void antlr::CharScanner::commit ( ) [inline, virtual]
virtual void antlr::CharScanner::consume ( ) [inline, virtual]
virtual void antlr::CharScanner::consumeUntil ( const BitSet set) [inline, virtual]

Consume chars until one matches the given set

virtual void antlr::CharScanner::consumeUntil ( int  c) [inline, virtual]

Consume chars until one matches the given char

virtual bool antlr::CharScanner::getCaseSensitive ( ) const [inline, virtual]
virtual bool antlr::CharScanner::getCaseSensitiveLiterals ( ) const [pure virtual]
virtual int antlr::CharScanner::getColumn ( ) const [inline, virtual]

Get the column the scanner currently is in (starts at 1)

virtual bool antlr::CharScanner::getCommitToPath ( ) const [inline, virtual]
virtual const std ::string& antlr::CharScanner::getFilename ( ) const [inline, virtual]

get the filename for the file currently used

virtual InputBuffer& antlr::CharScanner::getInputBuffer ( ) [inline, virtual]
virtual LexerSharedInputState antlr::CharScanner::getInputState ( ) [inline, virtual]
virtual int antlr::CharScanner::getLine ( ) const [inline, virtual]

Get the line the scanner currently is in (starts at 1)

int antlr::CharScanner::getTabSize ( ) const [inline]

Return the tabsize used by the scanner.

virtual const std ::string& antlr::CharScanner::getText ( ) const [inline, virtual]

return a copy of the current text buffer

virtual RefToken antlr::CharScanner::getTokenObject ( ) const [inline, virtual]
int antlr::CharScanner::LA ( unsigned int  i) [inline, virtual]
virtual RefToken antlr::CharScanner::makeToken ( int  t) [inline, protected, virtual]

Create a new RefToken of type t.

virtual unsigned int antlr::CharScanner::mark ( ) [inline, virtual]

Mark the current position and return a id for it.

virtual void antlr::CharScanner::match ( int  c) [inline, virtual]

See if input contains character 'c' throw MismatchedCharException if not.

virtual void antlr::CharScanner::match ( const BitSet b) [inline, virtual]

See if input contains element from bitset b throw MismatchedCharException if not

virtual void antlr::CharScanner::match ( const char *  s) [inline, virtual]

See if input contains string 's' throw MismatchedCharException if not

Note:
the string cannot match EOF
virtual void antlr::CharScanner::match ( const std::string &  s) [inline, virtual]

See if input contains string 's' throw MismatchedCharException if not

Note:
the string cannot match EOF
virtual void antlr::CharScanner::matchNot ( int  c) [inline, virtual]

See if input does not contain character 'c' throw MismatchedCharException if not

virtual void antlr::CharScanner::matchRange ( int  c1,
int  c2 
) [inline, virtual]

See if input contains character in range c1-c2 throw MismatchedCharException if not

virtual void antlr::CharScanner::newline ( ) [inline, virtual]

Used to keep track of line breaks, needs to be called from within generated lexers when a
is encountered.

CharScanner& antlr::CharScanner::operator= ( const CharScanner other) [private]
virtual void antlr::CharScanner::recover ( const RecognitionException ex,
const BitSet tokenSet 
) [inline, virtual]

called by the generated lexer to do error recovery, override to customize the behaviour.

virtual void antlr::CharScanner::reportError ( const RecognitionException e) [virtual]

Report exception errors caught in nextToken()

void CharScanner::reportError ( const std::string &  s) [virtual]

Parser error-reporting function can be overridden in subclass

virtual void antlr::CharScanner::reportWarning ( const std::string &  s) [virtual]

Parser warning-reporting function can be overridden in subclass

virtual void antlr::CharScanner::resetText ( ) [inline, virtual]
virtual void antlr::CharScanner::rewind ( unsigned int  pos) [inline, virtual]

Rewind the scanner to a previously marked position.

virtual void antlr::CharScanner::setCaseSensitive ( bool  t) [inline, virtual]
virtual void antlr::CharScanner::setColumn ( int  c) [inline, virtual]

set the column number

virtual void antlr::CharScanner::setCommitToPath ( bool  commit) [inline, virtual]
virtual void antlr::CharScanner::setFilename ( const std::string &  f) [inline, virtual]

Set the filename the scanner is using (used in error messages)

virtual void antlr::CharScanner::setInputState ( LexerSharedInputState  state) [inline, virtual]

set the input state for the lexer.

Note:
state is a reference counted object, hence no reference
virtual void antlr::CharScanner::setLine ( int  l) [inline, virtual]

set the line number

int antlr::CharScanner::setTabsize ( int  size) [inline]

set the tabsize. Returns the old tabsize

virtual void antlr::CharScanner::setText ( const std::string &  s) [inline, virtual]
virtual void antlr::CharScanner::setTokenObjectFactory ( factory_type  factory) [inline, virtual]

Set the factory for created tokens.

virtual void antlr::CharScanner::tab ( ) [inline, virtual]

Advance the current column number by an appropriate amount according to the tabsize. This method needs to be explicitly called from the lexer rules encountering tabs.

virtual int antlr::CharScanner::testLiteralsTable ( int  ttype) const [inline, virtual]

Test the token text against the literals table Override this method to perform a different literals test

virtual int antlr::CharScanner::testLiteralsTable ( const std::string &  txt,
int  ttype 
) const [inline, virtual]

Test the text passed in against the literals table Override this method to perform a different literals test This is used primarily when you want to test a portion of a token

virtual int antlr::CharScanner::toLower ( int  c) const [inline, virtual]

Override this method to get more specific case handling.

virtual void antlr::CharScanner::traceIn ( const char *  rname) [virtual]
virtual void antlr::CharScanner::traceIndent ( ) [virtual]

Methods used to change tracing behavior.

virtual void antlr::CharScanner::traceOut ( const char *  rname) [virtual]
virtual void antlr::CharScanner::uponEOF ( ) [inline, virtual]

This method is called by YourLexer::nextToken() when the lexer has hit EOF condition. EOF is NOT a character. This method is not called if EOF is reached during syntactic predicate evaluation or during evaluation of normal lexical rules, which presumably would be an IOException. This traps the "normal" EOF condition.

uponEOF() is called after the complete evaluation of the previous token and only if your parser asks for another token beyond that last non-EOF token.

You might want to throw token or char stream exceptions like: "Heh, premature eof" or a retry stream exception ("I found the end of this file, go back to referencing file").


Member Data Documentation

used to return tokens w/o using return val

Is this lexer case sensitive.

Used during filter mode to indicate that path is desired. A subsequent scan error will report an error as usual if acceptPath=true;

const int CharScanner::EOF_CHAR = EOF [static]

Input state, gives access to input stream, shared among different lexers.

std ::map< std ::string,int,CharScannerLiteralsLess> antlr::CharScanner::literals [protected]
const int CharScanner::NO_CHAR = 0 [static, private]

flag indicating wether consume saves characters

int antlr::CharScanner::tabsize [protected]

tab size the scanner uses.

std ::string antlr::CharScanner::text [protected]

Text of current token

Factory for tokens.


The documentation for this class was generated from the following files:
 All Classes Namespaces Files Functions Variables Typedefs Friends Defines