Module hw06

Package provided.tokenFW.tokenizer

Class StringTokenizer

java.lang.Object

provided.tokenFW.tokenizer.StringTokenizer

All Implemented Interfaces:

IStringTokenizer

Direct Known Subclasses:

ABCTokenizer

public class StringTokenizer
extends Object
implements IStringTokenizer

Special tokenizer that converts a stream of various types of tokens from StreamTokenizer and converts them into a stream of strings.

Author:

swong

Field Summary

Fields

Modifier and Type	Field	Description
protected Reader	_fileReader	Reader to use.
protected StreamTokenizer	_st	StreamTokenizer to use.
private ILogger	logger	logger to use
private Stack<String>	stack	Stack used for token push-back

Constructor Summary

Constructors

Constructor	Description
StringTokenizer(String inputFileName)	Initialize _st to read from a input Reader file with the given input file name.

Method Summary

Modifier and Type	Method	Description
String	getNextStringToken()	Get the next token in the stream
private String	makeNextStringToken()	Returns the next string in the stream.
void	putStringBack(String str)	Puts the given string back into the stream.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

logger

private ILogger logger

logger to use

stack

private Stack<String> stack

Stack used for token push-back

_st

protected StreamTokenizer _st

StreamTokenizer to use.

_fileReader

protected Reader _fileReader

Reader to use.

Constructor Details

StringTokenizer

public StringTokenizer(String inputFileName)
                throws FileNotFoundException

Initialize _st to read from a input Reader file with the given input file name.

Parameters:

inputFileName - the name of the input text file

Throws:

FileNotFoundException - When the input file could not be opened

Method Details

getNextStringToken

public String getNextStringToken()

Get the next token in the stream

Specified by:

getNextStringToken in interface IStringTokenizer

Returns:

the next token.

putStringBack

public void putStringBack(String str)

Puts the given string back into the stream. If nothing else is done, this would be the next token returned by getNextToken

Specified by:

putStringBack in interface IStringTokenizer

Parameters:

str - the token to push back into the tokenizer

makeNextStringToken

private String makeNextStringToken()

Returns the next string in the stream. Returns LF ("\n") at end of line. Returns null for EOF or error

Returns:

A string token or null if EOF