token.tokenizer
Class ABCTokenizer

java.lang.Object
  token.tokenizer.StringTokenizer
      token.tokenizer.ABCTokenizer

All Implemented Interfaces:

IStringTokenizer, ITokenizer

public class ABCTokenizer
extends StringTokenizer
implements ITokenizer

Tokenizer for the abc music language. getNextToken() returns next Token in stream. Does not recognize all possible language constructs and symbols (returned as "?" token). References:
java.util.regex.Pattern for regex guide
ASCII table for codes
abc music standard 1.6

Field Summary

static java.lang.String	ACCIDENTALS Allowable note accidentals
private java.util.Map<java.lang.String,java.lang.String>	keywords1 maps regex to their ASCII string counterparts.
private java.util.Map<java.lang.String,java.lang.String>	keywords2 keywords2 is searched second
private java.util.Vector<java.util.Map<java.lang.String,java.lang.String>>	keywordsVector Ordered sets of keywords to check.
private java.util.Stack<Token>	stack Push-back stack for tokens.
private ITokenFactory	tokFac Factory for the tokens

Fields inherited from class token.tokenizer.StringTokenizer

_fileReader, _st

Constructor Summary

ABCTokenizer(java.lang.String inputFileName)
Constructor for the class

Method Summary

Token	getNextToken() Return the next token, either from the stream or the last token if it was pushed back
Token	makeNextToken() Parse the strings down from the getNextStringToken further, separating out the abc keywords into tokens.
private java.lang.String	parseNote(java.lang.String sTok) Returns the first note string in the given string.
private java.lang.String	parseQuote(java.lang.String acc, java.lang.String sTok) Returns the rest of a string, incl.
void	putBack(Token t) Put the previously consumed token back into the token stream.

Methods inherited from class token.tokenizer.StringTokenizer

getNextStringToken, putStringBack

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

keywords1

private java.util.Map<java.lang.String,java.lang.String> keywords1

maps regex to their ASCII string counterparts. The key is the regex and the value is the keyword. Used for splitting strings up on the keywords. keywords1 is searched first

keywords2

private java.util.Map<java.lang.String,java.lang.String> keywords2

keywords2 is searched second

keywordsVector

private java.util.Vector<java.util.Map<java.lang.String,java.lang.String>> keywordsVector

Ordered sets of keywords to check.

tokFac

private ITokenFactory tokFac

Factory for the tokens

stack

private java.util.Stack<Token> stack

Push-back stack for tokens.

ACCIDENTALS

public static final java.lang.String ACCIDENTALS

Allowable note accidentals

See Also:

Constant Field Values

Constructor Detail

ABCTokenizer

public ABCTokenizer(java.lang.String inputFileName)
             throws java.io.FileNotFoundException

Constructor for the class

Parameters:

inputFileName - File to tokenize

Throws:

java.io.FileNotFoundException

Method Detail

getNextToken

public Token getNextToken()

Return the next token, either from the stream or the last token if it was pushed back

Specified by:

getNextToken in interface ITokenizer

putBack

public void putBack(Token t)

Description copied from interface: ITokenizer

Put the previously consumed token back into the token stream. Can only put back one token.

Specified by:

putBack in interface ITokenizer

makeNextToken

public Token makeNextToken()

Parse the strings down from the getNextStringToken further, separating out the abc keywords into tokens.

Returns:

The next Token in the stream

parseNote

private java.lang.String parseNote(java.lang.String sTok)

Returns the first note string in the given string.

Parameters:

sTok - The string to parse

Returns:

A String that is the first note in the given string

parseQuote

private java.lang.String parseQuote(java.lang.String acc,
                                    java.lang.String sTok)

Returns the rest of a string, incl. spaces until the next double quote

Parameters:

acc - Accumulated result string

sTok - the current string token

Returns:

the full quoted string