Module hw06
Package provided.tokenFW.tokenizer

Interface ITokenizer

All Known Implementing Classes:
ABCTokenizer
public interface ITokenizer
Extract and return an appropriate Token from some given source.

  • Method Summary

    Modifier and Type
    Method
    Description
    Token
    getNextToken()
    Return the next token.
    void
    putBack​(Token t)
    Put the previously consumed token back into the token stream.

  • Method Details

    • getNextToken

      Token getNextToken()
      Return the next token.
      Returns:
      the next token in the tokenizer

    • putBack

      void putBack​(Token t)
      Put the previously consumed token back into the token stream.
      Parameters:
      t - the token to put back into the tokenizer