Document License

 This work is licensed under the Creative Commons Attribution-Share Alike 3.0
 United States License. To view a copy of this license, visit 
 http://creativecommons.org/licenses/by-sa/3.0/us/
 or send a letter to Creative Commons, 171 Second Street, Suite 300,
 San Francisco, California, 94105, USA.

 
Contributors to this document

 Copyright (C) 2010 Texas Instruments Incorporated - http://www.ti.com/


TIesrFlex Grammar String Overview
---------------------------------

This document provides an overview of grammar string construction for the
TIesrFlex API.  The grammar string is provided to the ParseGrammar method of
TIesrFlex.  The grammar string defines the words and phrases that the TIesr
recognizer can recognize.


ParseGrammar Method
-------------------
The ParseGrammar method of the TIesrFlex API parses the grammar string prior to
building the binary grammar network and model file set.  The grammar string is
specified as a simple C character string argument.  Other ParseGrammar arguments
implement other options. Since the dictionary of word pronunciations used by
TIesrFlex may contain many different pronunciations of the same word, a
ParseGrammar argument can specify the maximum number of pronunciations in the
grammar network allowed for a word.  Some words in the grammar string may not be
in the dictionary.  In this case ParseGrammar will use a putative pronunciation
generated by the TIesrDT API.  However, if the word is in the dictionary, this
will most likely be the pronunciation desired.  ParseGrammar does provide an
argument that forces use of the putative pronunciation in addition to dictionary
pronunciations.  Finally, ParseGrammar will normally insert a model for silence
between each word of a phrase automatically.  An argument to ParseGrammar can
enable or disable the automatic insertion.  If it is disabled, then it is up to
the writer of the grammar string to manually account for any silence modeling
between words.  This will be discussed in detail later.


The Grammar String
------------------
The ParseGrammar grammar string argument specifies a null terminated C string
that represents the grammar that should be used to construct the binary grammar
network and acoustic model file set for use in subsequent recognition.  The
grammar must be specified in a single string.  The grammar string may consist of
single or multiple lines using newline as a line separator. Spaces, tabs and
newlines may be inserted in the string, but are ignored except as word
separators.  
The beginning of the grammar must consist of start(<NT>)., which defines the
beginning of the grammar.  In this part of the string, NT represents the
user-chosen starting non-terminal symbol name of the grammar.  All non-terminal
and terminal symbol names consist of alphabetic characters, dash(-), underscore(
_ ), or a single quote(). In general, non-terminal nodes should use some
convention, such as starting with an underscore, which makes them different from
terminal symbols which are words. 
Subsequent to the start of the grammar, the rest of the grammar consists of one
or more grammar rules. The first grammar rule must define the starting
non-terminal symbol. Each rule consists of a declaration of a non-terminal
symbol name on the left side, separated from its definition on the right side by
the characters  --->  (note that there is at least one space prior to and
after the "--->" symbols).  The right side contains the expansion of the
non-terminal symbol on the left side as a set of words and other non-terminal
symbol names. Words should normally consist of alphabetic characters, but may
include the other characters mentioned above. There are a few special exceptions
that will be discussed later. The expansion can use parentheses ( ) for
grouping, square brackets [ ] to indicate optional items, and the vertical bar
| to indicate an OR operation.  The right side must end with a period
character (.).  Words or non-terminal symbol names appearing next to each
other are assumed as combined with an AND operation, which takes precedence over
the OR operation.


Example grammars

Several examples of grammars follow.

This is a simple grammar recognizing either the word programs or "contacts.

     start(_S). _S ---> programs | contacts.

This grammar would recognize hello world or goodbye world.
     
     start(_S). _S ---> (hello | goodbye) world.

A more complex grammar would recognize seven or ten digits.  This is shown below
as multiple lines, but all text could be placed on one line, not using newline
characters.

     start(_Phonenumber).
     _Phonenumber ---> _Digit7 [_Digit3].
     _Digit7 ---> _digit _digit _digit _digit _digit _digit _digit.
     _Digit3 ---> _digit _digit _digit.
     _digit ---> zero | oh | one | two | three | four | five | six | seven |
                 eight | nine.


Important limitations 
---------------------
There are a few important restrictions on grammar construction.  These must be
followed carefully.  Most importantly, non-terminal symbols must not appear on
the left and right side of rules such as to cause a loop in the grammar.  A loop
grammar will eventually end up having the effect of defining a non-terminal
symbol in terms of itself. If the grammar is written in this way, then
ParseGrammar will return a failure to parse the grammar.  An example of a loop
grammar is shown below.  This grammar will not parse. Notice that if in the
_Loop rule the _NextRule non-terminal is substituted with the expansion of its
definition, then _Loop is defined in terms of itself.

     start(_Loop).
     _Loop ---> this is a _NextRule.
     _NextRule ---> _Loop grammar.
 
Each non-terminal symbol name must appear only once on the left hand side of all
rules.  If it appears more than once on the left side, then it causes a
re-definition of the rule. This will not cause ParseGrammar to fail, but will
result in a grammar that was not intended. 

Obviously, terminal symbol words and non-terminal symbols should not be the
same.  This is the reason that a convention of starting non-terminal symbols
with an underscore may be helpful.

Rule lines may be of any arbitrary length.  


Special Words
-------------

TIesrFlex defines two types of special silence pseudo-words.  


The _SIL word
-------------
One pseudo-word is the _SIL word.  This word represents an arbitrary amount of
silence.  When the ParseGrammar method argument specifies automatic insertion of
silence between words, then this pseudo-word is inserted as an optional word at
the beginning of the utterance, end of the utterance, and between each word. 
For example, given this grammar:

     start( _Name). _Name ---> John Doe.

Then the effect of automatic silence insertion will be to implement a grammar
that really looks like:

     start( _Name). _Name ---> [ _SIL ] John [ _SIL ] Doe [ _SIL ].

Typically this is what the grammar designer desires, since this models optional
pause prior to and after words.


_SIL_* words
------------
The other set of pseudo-words is given by _SIL_*, where * is any string of ascii
characters.  The purpose of these pseudo-words is to easily index what could be
complex recognition results.  Upon successful completion of recognition, the
TIesr recognizer returns the words it recognizes.  However, in many cases, one
is not interested in the sequence of words recognized, but rather which phrase
was actually spoken.  When a _SIL_* pseudo-word is placed in the grammar, it
acts just like insertion of _SIL from the acoustic perspective, but TIesr
returns the pseudo-word that was in the path that was recognized.  For example,
the following grammar may need to be recognized, and action taken based on which
phrase was spoken.  Since the grammar is somewhat complex and defined
dynamically during running of the application, it can be involved to determine
which phrase was spoken by just examining which words were recognized. However,
it is relatively easy to detect the ending pseudo-word.

  start(_Select).
  _Select ---> [ [please] go to] ( _Selection1 _SIL_A | _Selection2 _SIL_B.)
  _Selection1 ---> an arbitrary run time phrase.	
  _Selection2 ---> a different arbitrary run time phrase.

The intent is to normally use these pseudo-word models at the end of each
phrase.  In this way it is easier to locate the pseudo-word and determine the
spoken phrase.


Helpful Hint

It is best to keep the grammar as simple as possible.  Excessive use of optional
phrases,  OR clauses, and grouping within a single rule can be confusing to read
and can easily lead to mistakes in specifying a grammar. In the event that
recognized phrases seem to be inconsistent with the grammar, a tool is available
to examine the detailed output grammar network structure that TIesrFlex creates.
