Public Types | Public Member Functions | Public Attributes | List of all members
zorba::Tokenizer::State Struct Reference

A State contains inter-Tokenizer state, currently the current token, sentence, and paragraph numbers. More...

#include <zorba/tokenizer.h>

Public Types

typedef Tokenizer::size_type value_type
 

Public Member Functions

 State ()
 Default constructor. More...
 

Public Attributes

value_type para
 Paragraph number. More...
 
value_type sent
 Sentence number. More...
 
value_type token
 Token number. More...
 

Detailed Description

A State contains inter-Tokenizer state, currently the current token, sentence, and paragraph numbers.

Definition at line 54 of file tokenizer.h.

Member Typedef Documentation

Definition at line 55 of file tokenizer.h.

Constructor & Destructor Documentation

zorba::Tokenizer::State::State ( )

Default constructor.

Member Data Documentation

value_type zorba::Tokenizer::State::para

Paragraph number.

Definition at line 59 of file tokenizer.h.

value_type zorba::Tokenizer::State::sent

Sentence number.

Definition at line 58 of file tokenizer.h.

value_type zorba::Tokenizer::State::token

Token number.

Definition at line 57 of file tokenizer.h.


The documentation for this struct was generated from the following file: