token.tokenizer
Interface ITokenizer

All Known Implementing Classes:

ABCTokenizer

public interface ITokenizer

Extract and return an appropriate Token from some given source.

Method Summary

Token	getNextToken() Return the next token.
void	putBack(Token t) Put the previously consumed token back into the token stream.

Method Detail

getNextToken

Token getNextToken()

Return the next token.

putBack

void putBack(Token t)

Put the previously consumed token back into the token stream. Can only put back one token.