Overall Procedure:
==================
create description of recognizer. Find samples to train on.
train using 1/0 targets. re-estimate parameters using soft targets.


	if not exist, create corpora file
	if not exist, create cull files for all corpora using cull5.cslush

	create digit.info files (train, dev, dev1, test, force)
	create digit.vocab
	create digit.parts

	run find_files.cslush for training, dev*, and test

	run categories.cslush to determine categories to use and create desc
	run hscript to create digit.0 and hmm files

	run gen_catfiles.cslush to create category-label files

    make sure that all data is there; if not, 
  			revise digit.parts and/or digit.vocab 
			run gen_catfiles
			run categories
			run hscript
			run revise_desc
    	or 
			run revise_desc to create ties and revise desc files
			run hscript

	run pickframes.cslush
	run genvec.cslush
	run checkvec.cslush
	run shuffle

	run nntrain
	(run nnscore)
  * ** run find_best.cslush

	----
	forced alignment
	generate vector file using hmm_embed.cslush
	check vector file using hnncheckvec
	re-train using hnntrain
	run find_best.cslush
	----

	REPEAT the forward-backward step


	run find_best.cslush on dev2 set, eventually test set

  * run browse.cslush on .ali file


Specifics:
==========

corpora file
------------
xxx

digit.desc
----------
The description files describes all aspects of the recognizer,
including the size of feature vectors, state-transition 
probabilities, and labels for the categories used in recognition.

At this point in time, the description file (digit.desc, in this
example) must be created "by hand"; there are not yet any automatic
scripts to generate this file. This file can be created as
described in the "hscript" page.


digit.info
--------------
The info file specifies all information needed to find samples
for training, cross-validation, or testing. It includes the
category (train, development, or test), how to partition the
data into the category, the basename for the recognizer, the
minimum number of samples needed, and corpus-dependent information.
The corpus-dependent information includes the corpus name,
the directory in which to write time-aligned categories, how
many samples are desired from each category, the remapping program
to use (if any), and other information. This file is used by
find_data.cslush to select and remap files, and it is used by
pickframes.cslush to select frames for training.


find_data.cslush
------------
Given a corpus to use, we need to select the files in this corpus
that we want to use for training, cross-validation, or testing.
If we have data that has been hand-labeled at the phonetic level
(including time alignment), then we can train our recognizer using 
these data.  However, our recognizer uses categories that are 
context-dependent and at the sub-phonetic level. Therefore, we need 
to convert the phonetic-level labels into context-dependent 
sub-phonetic category labels. The script find_data.cslush selects
the files to use and automatically does the conversion for us.

Alternatively, if we have text transcriptions but no hand-labeled
phonetic data, we can use find_data.cslush to select the files
we want and, instead of converting phonetic-labled files, perform
forced-alignment to generate automatic phonetic-labeled data.

Specifically, mapdigit.cslush maps hand-labeled time-aligned data (in
the "lola" format) to time-aligned context-dependent categories (also
in the "lola" format). It also changes some phonemes to unique 
allophones, depending on the context of surrounding phones. For
example, the last vowel in the word "seven" is mapped to the 
(application-specific) allophone "^2", because this vowel is hard
to label consistently using only Worldbet symbols.

usage: mapdigit.cslush <description file> <directory for mapped files>
input: description file and hand-labeled files from numbers corpus
	   (assumed to be in directory called "./phn")
output: .nnphn files in the specified directory for mapped files


hscript
-------
The user has specified all relevant information in the description 
file, but other programs find it easier to process data if the 
relevant info is in a different format. This program computes all 
formats necessary for later programs, based on the user's description 
file.

usage: hscript
input:  description file
output: digit.rr, digit.0, and digit.list

pickdigit.cslush
-------------
finds frames to train on
input: digit.rr, digit.0, and digit.list
output: digit.pick, digit.ff
Purpose: We need to find frames to train on. We have the
description of the recognizer and we have the labeled categories,
now we need to select frames for training. Often we don't
want to train on all avaiblle frames, so we speicify the maximum
number of frames we want. This script locates the data for
training for us, and writes in in a format to be read by genvec.

genvec.cslush
----------
creates vectors for training
input: digit.pick
output: digit.vec
Purpose: given the frames we want to train on, we need to 
compute the paramters at each frame that will be used to train
the network. We compute all these parameters and put them in
a .vec file for nntrain

nntrain
-------
trains neural network using binary targets
input: digit.vec
output: nnet.X, where X = {0, 5, 10, 15, ...}
Purpose: this proram computes the weights to be used in the
1/0 neural network using the backprop method. Targets are 0 and
1, and the input features are taken from digit.vec.

hmmembed.cslush
------------
prepares for embedded training by assigning probablistic targets
to the same data as before.
input:
output:
Purpose: before training on soft targets, we need to determine
the values of the soft targets. This program does this.

hnntrain
--------
trains neural network using probabalistic targets
input: digit.1
output: nnet.X, where X = {0, 5, 10, 15, ...}
Purpose:
