Module hw06

Package provided.tokenFW.tokenizer

package provided.tokenFW.tokenizer

Tokenizer and support classes to read the text stream from an ABC music text file and generate a stream of tokens from it.

Author:

Stephen, Scott Rixner

Interface Summary

Interface	Description
IStringTokenizer	Tokenizer that only returns string tokens.
ITokenizer	Extract and return an appropriate Token from some given source.

Class Summary

Class	Description
StringTokenizer	Special tokenizer that converts a stream of various types of tokens from StreamTokenizer and converts them into a stream of strings.