NICO (Neural Inference COmputation) tool-kit

Category Intelligent Software>Neural Network Systems/Tools

Abstract The NICO Toolkit is an artificial neural network (ANN) toolkit designed and optimized for speech technology applications.

Although it is mainly intended for, and originally developed for speech recognition applications, the NICO toolkit is a general purpose toolkit for constructing artificial neural networks and training with the back-propagation learning algorithm.

The network topology is very flexible. Units are organized in groups and the group is a hierarchical structure, so groups can have sub-groups or other objects as members.

This makes it easy to specify multi-layer networks with an arbitrary connection structure and to build modular networks.

Recurrency and time-delays --

Recurrency and time-delays are constrained only by the condition that the activation of a unit cannot be dependent on its own activity at the present or future times (this is checked automatically every time the user connects between two (2) groups or units).

A convenient feature is that the user can specify Not only time-delay connections but also look-ahead connections.

All connections are converted to time-delays when the network runs, but this is completely hidden from the user.

Developed to solve a particular problem --

The NICO toolkit was developed to solve a particular problem (phoneme probability estimation for automatic speech recognition) and the focus was Not on research in the field of artificial neural networks.

There are No fancy graphical tools to monitor network behavior or characteristics.

But if you want to solve a real problem with a large database (millions of samples) and possibly large networks (e.g. 500,000 connections or more), then this toolkit is probably still a good choice as it is optimized for fast training and evaluation on large networks.

The communication with external data is handled conveniently by stream objects in the network. A network may have multiple input and output streams.

The streams perform the normalization of the data to values in the range suitable for ANN computation. Streams can read data in several different file formats.

Training algorithm --

The only training algorithm available is a fast implementation of the back-propagation learning algorithm.

It is optimized for high performance back-propagation through time, i.e., for training of networks with recurrency and/or time-delay windows.

The reason that none of the theoretically more advanced algorithms (like quasi-newton, conjugate-gradient, etc.) are available is that the NICO toolkit was developed for training large networks with the training typically terminated before the minimum error is reached (to prevent over-training) and weight updating many times per epoch.

For this type of problems, back-propagation with momentum is a good choice.

Units of several different types can be arbitrarily mixed in a neural network (NN).

Some of the unit types are:

1) Sigmoid units;

2) Tanhyp (hyperbolic tangent) units;

3) Linear units;

4) Multiplication units;

5) Exponential units, ex;

6) Inverter units, 1/x;

7) Environment units (takes the activate system environment);

8) File filter units (Boolean unit that checks the filename of the input data); and

9) Other unit types with, for example x2, sigma-pi, softmax, and radial basis activation functions can be modeled by combining the primitive unit types above.

Many different error functions are supported and can be arbitrarily mixed in a network.

Some of the error functions are:

1) Mean square error;

2) Cross entropy; and

3) Absolute value.

What the NICO toolkit does Not do --

1) No Graphical User Interface (GUI) for viewing and building networks;

2) No support for other types of NN training than back-propagation;

3) Weight-sharing is Not possible in the current implementation; and

4) The tools provided compute the activities of the ANN. A speech recognizer based on the hybrid Hidden Markov Model (HMM)/ANN speech recognition paradigm also requires HMM software that is Not included in this toolkit.

System Requirements

Contact manufacturer.

Manufacturer

Manufacturer Web Site NICO

Price Contact manufacturer.

G6G Abstract Number 20682

G6G Manufacturer Number 104260