Module hw06
Package provided.abcMusic.parser.tokenizer

Class ABCTokenizer

java.lang.Object
provided.tokenFW.tokenizer.StringTokenizer
provided.abcMusic.parser.tokenizer.ABCTokenizer
All Implemented Interfaces:
IStringTokenizer, ITokenizer
public class ABCTokenizer extends StringTokenizer implements ITokenizer
Tokenizer for the abc music language. getNextToken() returns next Token in stream. Does not recognize all possible language constructs and symbols (returned as "?" token). Tokens are the atomic psemantic pieces of a language, such as Note, repeat start, measure, chord, etc. The token ids (names) returned are:

EOF - end of file
"|:" - repeat start
":|" - repeat end
"|1" - first repeat start
"[1" - alternative first repeat start
"|2" - second repeat start
"[2" - alternative second repeat start
"[|" - thick-thin double bar line
"|]" - thin-thick double bar line
":|2" - alternative second repeat start
"||" - double measure bar
"::" - short hand for repeat end-start ":||:"
"|" - Measure bar
"[") - Chord start
"]" - Chord end
"-" - tie
"<" - the previous note is dotted, the next note halved
">" - the previous note is halved, the next dotted
"{" - Grace begin
"}" - Grace end
"%" - Comment
"[Header ID]" - where [Header ID] is a single letter from {ABCDEFGHIKLMNOPQRSTWXZ}
"Note" - a single note whose lexeme contains the name, accidentals, octave and duration
"Tuplet" - the lexeme is the number of subsequent Notes comprise this tuplet.
"Quote" - lexeme is the quoted text
"?" - unknown language construct.

References:
java.util.regex.Pattern for regex guide
ASCII table for codes
abc music standard 1.6

  • Field Details

    • keywords1

      private Map<String,​String> keywords1
      maps regex to their ASCII string counterparts. The key is the regex and the value is the keyword. Used for splitting strings up on the keywords. keywords1 is searched first

    • keywords2

      private Map<String,​String> keywords2
      keywords2 is searched second

    • keywordsVector

      private Vector<Map<String,​String>> keywordsVector
      Ordered sets of keywords to check.

    • tokFac

      private ITokenFactory tokFac
      Factory for the tokens

    • stack

      private Stack<Token> stack
      Push-back stack for tokens.

    • ACCIDENTALS

      public static final String ACCIDENTALS
      Allowable note accidentals
      See Also:
      Constant Field Values

  • Constructor Details

  • Method Details

    • getNextToken

      public Token getNextToken()
      Return the next token, either from the stream or the last token if it was pushed back
      Specified by:
      getNextToken in interface ITokenizer
      Returns:
      the next token in the tokenizer

    • putBack

      public void putBack​(Token t)
      Push token back into the stream
      Specified by:
      putBack in interface ITokenizer
      Parameters:
      t - the Token you need to put back in the stream.

    • makeNextToken

      private Token makeNextToken()
      Parse the strings down from the getNextStringToken further, separating out the abc keywords into tokens.
      Returns:
      The next Token in the stream

    • parseNote

      private String parseNote​(String sTok)
      Returns the first note string in the given string.
      Parameters:
      sTok - The string to parse
      Returns:
      A String that is the first note in the given string

    • parseQuote

      private String parseQuote​(String acc, String sTok)
      Returns the rest of a string, incl. spaces until the next double quote
      Parameters:
      acc - Accumulated result string
      sTok - the current string token
      Returns:
      the full quoted string