- All Known Implementing Classes:
ABCTokenizer
,StringTokenizer
public interface IStringTokenizer
Tokenizer that only returns string tokens. This is a specialization of ITokenizer for strings.
- Author:
- swong
-
Method Summary
Modifier and TypeMethodDescriptionReturn the next token.void
putStringBack(String t)
Put the previously consumed token back into the token stream.
-
Method Details
-
getNextStringToken
String getNextStringToken()Return the next token.- Returns:
- The next token in the tokenizer
-
putStringBack
Put the previously consumed token back into the token stream. Can only put back one token.- Parameters:
t
- the token to push back into the tokenizer
-