AI Trilogy Problem

Maybe you or someone in your company can relate to the following problem. Your company is formulating some mixture: maybe a drug, a bonding compound, an animal feed, foodstuff or beverage, plastic or other packaging material, cosmetic or perfume, lubricating compound, cleaning compound or solvent, paint, insecticide or herbicide, a manufactured item or process, or even problems such as marketing strategies. You were given the job of finding the optimum mixture of components or process variables, the combination of components and processing methods that produces the best product. (Components are either absolute values or the percentages of each of the elements of the mixture and the process variables are factors in the building process such temperature, concentration, curing time, etc.)

Sound familiar? You don’t even know how the various factors interrelate, much less how to discover the best ones. What do you do? We call this problem the "AI Trilogy Problem", and we believe it is becoming a very common problem in industry today! Fortunately it is quite easy and inexpensive to solve if you have the right tools and the "know how".

The AI Trilogy™ is a new packaging of our most popular business and scientific software. The AI Trilogy contains the NeuroShell Predictor, NeuroShell Clasifier, and GeneHunter together as one package. The NeuroShell Run-Time Server has also been added to the trilogy to make it possible for the three products to work together, either in an Excel spreadsheet or in a computer program.

So how does the AI Trilogy help you solve the AI Trilogy Problem? There are two steps you need to accomplish.

1. Model the Process

First you model the process so that the interrelationships between the components and process variables are in the model. To build the model you create and train a neural network using historical or test data. For this step you use either NeuroShell Predictor or NeuroShell Classifier depending on the type of the network. Networks with continuous outputs are built with the Predictor. Networks with category outputs are built with the Classifier. Examples of the category outputs are "True"/"False", "0"/"1", "Yes"/"No", "Cancer"/"Benign", "Buy"/"Hold"/"Sell", and so forth.

2. Optimize the Model

Next you optimize the model to find those special combinations (among the vast number possible) of inputs that produce the kind of product you need. GeneHunter performs the model optimization. The fitness function to be optimized is your model (neural network) output. For this step the NeuroShell Run-Time Server provides all the necessary support to compute the fitness function value outside of the Predictor/Classifier.

Microsoft Excel

The Microsoft Excel worksheet is the best common ground on which the model optimization can be performed due to its capabilities to call both GeneHunter and NeuroShell Run-Time add-ins. However, if you prefer to write a computer program to do the optimization, GeneHunter and the Runtime Server provide for this.


Components

The AI Trilogy components are: NeuroShell Predictor, NeuroShell Classifier, NeuroShell Run-Time Server, GeneHunter, two problem examples, and this documentation. By default, the AI Trilogy setup puts each component in its own folder on the hard disk. In addition, each component has its own folder in the Start button/Programs menu.

Component | Default setup folder | Start button/Programs menu
NeuroShell Predictor | C:\AI Trilogy\Predictor | NeuroShell Predictor
NeuroShell Classifier | C:\AI Trilogy\ Classifier | NeuroShell Classifier
NeuroShell Run-Time Server | C:\AI Trilogy\ Run-Time | NeuroShell Run-Time Server
GeneHunter | C:\AI Trilogy\GeneHunter | GeneHunter
AI Trilogy examples | C:\AI Trilogy\Examples | N/A
AI Trilogy help | C:\AI Trilogy | AI Trilogy


NeuroShell Predictor

The NeuroShell® Predictor is a software program designed to simplify the creation of a neural network application to solve your forecasting and pattern recognition problems. (If you need software to assist with classification problems such as whether or not to approve a loan application, the NeuroShell® Classifier is more appropriate to the task.

How Do I Build a Prediction Neural Network?

Neural network technology mimics the brain's own problem solving process. Just as humans apply knowledge gained from past experience to new problems or situations, a neural network takes previously solved examples to build a system of "neurons" that makes new decisions, classifications, and forecasts.

To train the network you must have a data set containing sample facts or parameters with the corresponding answers or results. The data set used for training can be obtained using historical problem data in which the outcomes are known, or by creating sample problems and solutions with the help of experts. Once the training process is completed, the network should be able to classify or predict from new inputs. Ward Systems Group provides free technical support including assistance with how to build your particular application.

Neural nets deal with data in a rows and columns format. For example, suppose you have a restaurant, and you are interested in predicting the restaurant's monthly sales. The following is an example of how to use a neural network to make such sales predictions.

First, lay out in a tabular format in a word processor or spreadsheet the historical data you need, such as the season, number of special events scheduled in the town, the number of ads you have placed, and the cost of those ads. Each month's data should be included in a row in the table. The input variables that affect your prediction of monthly sales are columns in the table. You can use any variables that you think affect sales.

Our neural nets "learn" to recognize patterns from data presented as above. After this learning, they are ready to make predictions for you.

To create a prediction system:

1. Select a data file that contains examples of what you are trying to predict.

2. Tell the Predictor which columns are inputs (independent variables) and which column you are trying to predict (the output or dependent variable). If you need to predict more than one variable, create two models but use the same data file for both.

3. Choose a training strategy (either neural or genetic).

4. Train the network.

5. Apply the trained network to existing data or new data to obtain the network’s predictions.

More details about the NeuroShell Predictor are available from the Predictor Help file (Nspred.hlp) that is located in the AI Trilogy\Predictor directory if you installed the AI Trilogy in the default directory.


NeuroShell Classifier

The NeuroShell® Classifier is a software program designed to simplify the creation of a neural network application to solve your classification problems. (If you need software to assist with forecasting and prediction problems such as predicting a price, quality level, or chemical strength, the NeuroShell® Predictor is more appropriate for the task.

How Do I Build a Classification Neural Network?

Neural network technology mimics the brain's own problem solving process. Just as humans apply knowledge gained from past experience to new problems or situations, a neural network takes previously solved examples to build a system of "neurons" that makes new decisions, classifications, and forecasts.

To train the network you must have a data set containing sample facts or parameters with the corresponding answers or results. The data set used for training can be obtained using historical problem data in which the outcomes are known, or by creating sample problems and solutions with the help of experts. Once the training process is completed, the network should be able to classify or predict from new inputs. Ward Systems Group provides free technical support including assistance with how to build your particular application.

Neural nets deal with data in a rows and columns format. For example, suppose you are a school counselor, and you are interested in classifying the success of students based on their TV and study habits. The following is an example of how to use a neural network to make such classifications.

First, lay out in a tabular format in a word processor or spreadsheet the historical data you need, such as the student number (not used in the model), number of hours of TV watched each week, number of hours of study time each week, and a classification as to whether the student was on the honor roll. Each student’s data should be included in a row in the table. The input variables that affect a classification of honors are columns in the table. You can use any variables that you think affect a student earning the grades to get on the honor roll.

Our neural nets "learn" to recognize patterns from data presented as above. After this learning, they are ready to make classifications for you.

To create a classification system:

1. Select a data file that contains examples of what you are trying to classify.

2. Tell the Classifier which columns are inputs (independent variables) and which column contains the classification (the output or dependent variable).

3. Choose a training strategy (either neural or genetic).

4. Train the network.

5. Apply the trained network to existing data or new data to obtain the network’s classifications.

More details about the NeuroShell Classifier are available from the Classifier Help file that is located in the AI Trilogy\Classifier directory if you installed the AI Trilogy in the default directory.


NeuroShell Run-Time

NeuroShell® Run-Time Server is a series of programs, Microsoft Excel Add-Ins, OCXs and DLLs that allow you to run your networks created by either the NeuroShell® Predictor or the NeuroShell® Classifier. Using the Run-Time facilities, you can apply these networks without using the NeuroShell® Predictor or the NeuroShell® Classifier programs. These facilities allow you to fire networks from a simplified interface, from your own computer programs, or from Excel spreadsheets.

There are six distinct components available in the NeuroShell® Run-Time Server:

NeuroShell Fire Program (NSFIRE.EXE) - NOT distributable

NeuroShell Fire Excel Add-In (NSFIRE.XLA) - NOT distributable

NeuroShell Run OLE DLL (NSRUN.DLL) - distributable

NeuroShell Run ActiveX Control (NSRUN.OCX) - distributable

NeuroShell Run Excel Add-In (NSRUN.XLA) - distributable

NeuroShell Run Wrapper DLL (NSWRAP.DLL) - distributable

Although the DLLs and the OCX are compatible with many of the programming languages available, we strongly recommend that you use Visual Basic for you applications, since it can do so much for you. We also recommend that you use the OCX over the DLLs if possible.

The components labeled above as distributable may be sold or otherwise distributed outside of your company or government agency as part of your computer programs or Excel spreadsheets. Those who you distribute to, however, MAY NOT use the distributable components for any purpose other than to run the application you have provided them. The components labeled NOT distributable may only be distributed to employees of your company or government agency, but not to anyone else, including employees of parent or subsidiary companies or agencies.


GeneHunter

GeneHunter is a powerful software solution for optimization problems which utilizes a state-of-the-art genetic algorithm methodology. GeneHunter includes an Excel Add-In which allows the user to run an optimization problem from an Excel spreadsheet, as well as a Dynamic Link Library of genetic algorithm functions that may be called from programming languages such as Microsoft® Visual Basic or C.

What Are Genetic Algorithms?

Genetic algorithms (GAs) seek to solve optimization problems using the methods of evolution, specifically survival of the fittest. In a typical optimization problem, there are a number of variables which control the process, and a formula or algorithm which combines the variables to fully model the process. The problem is then to find the values of the variables which optimize the model in some way. If the model is a formula, then we will usually be seeking the maximum or minimum value of the formula. There are many mathematical methods which can optimize problems of this nature (and very quickly) for fairly "well-behaved" problems. These traditional methods tend to break down when the problem is not so "well-behaved."

Evolution and Genetic Algorithms

Before describing how a GA can be applied to an optimization problem, let us draw the evolutionary parallel. The theory is that a population of a certain species will, after many generations, adapt to live better in its environment.

For example, if the species is an animal that lives mainly in a swampy area, it may eventually evolve with webbed feet. The reason is that the members of this population, which we call individuals, will tend to die if they are poor swimmers which cannot easily get food, and live to reproduce if they are good swimmers. The offspring of two good swimmers will probably be good swimmers because they will usually carry genetic traits of their parents, such as slight webbing between their toes. These genetic traits are carried in the chromosomes of the individuals.

How Does GeneHunter Work?

GeneHunter solves optimization problems in the same way. It will create a population of possible solutions to the problem. The individuals in this population will carry chromosomes which are the values of variables of the problem.

GeneHunter actually solves your problem by allowing the less fit individuals in the population to die, and selectively breeding the most fit individuals. The process is called selection, as in selection of the fittest. GeneHunter takes two individuals and mates them (crossover). The offspring of the mated pair will receive some of the characteristics of the mother and some of the father.

In nature, offspring often have some slight abnormalities, called mutations. Usually, these mutations are disabling and inhibit the ability of the offspring to survive, but once in a while, they improve the fitness of the individual. GeneHunter occasionally causes mutations to occur.

As GeneHunter mates fit individuals and mutates some, the population undergoes a generation change. The population will then consist of offspring plus a few of the older individuals which GeneHunter allows to survive to the next generation. These are the most fit in the population, and we will want to keep them breeding. These most fit individuals are called elite individuals. After dozens or even hundreds of "generations," a population eventually emerges wherein the individuals will solve the problem very well. In fact, the most fit (elite) individual will be an optimum or close to optimum solution.


General Modeling Considerations

For most AI Trilogy applications, the relationships between the components, process variables, and the quality or quantity of the product are not already known mathematically. By this we mean that there is usually no known formula in which one could enter the "inputs" to the process and compute the desired "output."

Inputs (Independent Variables)

The inputs would usually be one or more of the following:

1. A given mixture of components of the product, e.g., the percentages, strengths, or absolute values of chemicals or other materials that make up a particular batch of the compound (the product).

2. A given set of settings of the "knobs" that control the process of making the compound, e.g., the amount of pressure applied, the length of time the compound is allowed to cure, etc.

3. A given set of environmental settings, e.g., the temperature and humidity at which the compound is made.

The inputs used will vary from problem to problem, but will generally include only those things known or suspected to influence the quality of the resulting product. Furthermore, you would not need to include any input which can never be varied.

Output (Dependent Variable)

The output will usually be one or more of the following:

1. The quantity of the resulting product

2. The quality of the resulting product

3. The category of the resulting product

If the Formula is Known

If an appropriate formula is already known that takes the inputs as parameters and computes the output, then it can be used instead of building a neural network model. If a formula is known, but not considered accurate enough, then the formula results can become one of the inputs to a neural network model. We call this letting the neural network "stand on the shoulders of giants."


Exemplars

A neural network model of the process is constructed with historical examples or observations which we will sometimes call "exemplars" or "training patterns". The neural network will "learn" these exemplars in order to construct a model of the process.

Each exemplar represents one way of building the product, and is usually obtained by actually building the product under measured conditions. Therefore, an exemplar will consist of all the values of the inputs (how they were set) when a particular version of the product was made. The exemplar also needs to contain the measured output: how good the product was with these input settings, or how much was produced, or what quality category it fell into, etc.

You may already have plenty of exemplars from historical records of building the product in the past. You can use these records as long as they:

1. are accurate,

2. contain all of the inputs and outputs you feel you need,

3. vary as widely as you feel necessary, and

4. are plentiful.

The items above bear some further explanation.

1. Accuracy. Neural networks are "fuzzy" models that can tolerate a certain amount of small inaccuracies (some call it noise) especially if there are numerous examples that, taken together, will tend to counteract the inaccuracies. Nevertheless, major data inaccuracies, or too many minor ones, may render the final model inaccurate, too. Only you, with your domain knowledge of the problem, can decide if the data is accurate enough.

2. Comprehensive. The selection of input and output variables is critical to good model building. Garbage in, garbage out, as they say. The model can be no better than what you give it as inputs. If you don’t have as inputs the factors that really affect the product, then the resulting model will not be as good as it could be. If the model is not as good as it could be, then any optimization looking for optimal factors will produce factors that are no really optimal. The NeuroShell Predictor and Classifier both provide features to help you sort out the effectiveness of inputs in case you aren’t sure.

3. Variety. The neural net must learn on a set of training patterns that are scattered widely across the "domain" of the problem. An example is worth a 1000 words. Let’s say your major inputs are zinc and aluminum. Suppose that zinc may reasonably be in the range of 10% to 30% of the product, and aluminum must be between 20% and 50% (the rest of the product consists of some sort of filler). The neural net model cannot "know" about a combination of these components (call the combination X) unless it "learns" about combinations that are close to or around X.

So here is a bad set of {zinc, aluminum} exemplars:

{10, 20}, {12, 25}, {15, 27}, {10, 25}

That set is bad because it teaches the neural net nothing about zinc over 15% and aluminum over 27%.

The following is bad, too:

{10, 20}, {20, 20}, {30, 20}, {10, 30}, {10, 40}, {10, 50}

At first glance you might conclude that each of the two variables is represented throughout its range, but upon closer examination we see that there is nothing in the neighborhood of {30, 50}.

4. Not only must there be exemplars of various combinations, but there must be as many of them as needed to make sure that no large "gaps" occur in the problem domain. In any case, there should be far more of them than there are inputs. A good rule of thumb is a MINIMUM of 10 exemplars for every input, but 20 or 30 is much better. In practice this is often hard or expensive to achieve.

If you don’t already have plenty of the right exemplars, you will need to make them. This may require actual test runs in your "plant", during which all of the inputs and outputs are carefully measured and controlled.

The most convenient way to collect the exemplars is to gather the data in an Excel spreadsheet. The data can then easily be save as a .csv file, a type of text file that both the NeuroShell Predictor and NeuroShell Classifier will read.


Input Considerations

As already mentioned, you need to select inputs that affect the output. The ones selected must vary widely within acceptable ranges.

Inputs must be numeric, not text, because neural nets accept only numbers. If you have text like "low", "medium", and "high" then code those as 1, 2, and 3. Be sure to keep the numbers monotonic, i.e., as the concept increases the numbers increase.

Binary concepts (true,false or yes,no) can be coded 0,1 or 1,2, but binary numbers should generally be avoided when possible.

Sometimes binary coding is unavoidable. Suppose you want an input value that is the code for which of your company’s manufacturing plants made a particular part. The plants are East Coast, Southern, Northern, and West Coast. You might be tempted to make this your input:

1=East Coast
2=Southern
3=Northern
4=West Coast

This would be OK if there are only two plants, but with four the increasing values don't mean an increasing input concept to the net.

If, however, you were trying to represent the fact that some plants are more capable than others, and the capabilities were listed most to least or least to most, then your coding is OK.

Otherwise you would be forced to use four variables, one for each plant, which are either one or zero.


Output Considerations

The network output will be either a continuous value or a category.

Examples of continuous values are:

Quality ranked from 1 to 10 (it doesn’t matter whether 1 or 10 represents best)

Probability that the product is good, from 0 to 1

Time it takes the capsule to dissolve

Mean time to failure

Amount of heat buildup

Expected profit (e.g., the product is a business strategy)

Amount of milk expected from your herd (see Cow Feed Optimization Example)

The pH value (see Chemical Reactor Example)

If your output is a continuous value, as most will be, you would use the NeuroShell Predictor to model it.

Examples of categories include:

Product is "acceptable" or "not acceptable"

Product is "acidic" or "basic"

Product is "bland", "salty", "sweet", "bitter", or "sour"

You would usually use categories when there is no single continuous value that determines the result. For example, you may be able to measure pH, in which case the "acid" or "basic" category could be posed as a continuous problem instead. If your output is a category, you will use the NeuroShell Classifier to model the problem. The Classifier presents an output as a set of probabilities, one for each category.

The NeuroShell Predictor builds only one output per model. If your problem needs several outputs, you can usually just build several models. GeneHunter can do optimizations where several outputs are optimized simultaneously.

Please consult the NeuroShell Predictor or NeuroShell Classifier Help files for details on their use. For advanced topics and understanding, you will want to consult our special web site www.ward.net (this is NOT our sales website!)


Important Network Considerations

The NeuroShell Predictor does two types of modeling called "neural" and "genetic". The neural method is best suited to the AI Trilogy problem, because it can extrapolate. In other words, it can produce output numbers above or below any that it has "seen" during training. Therefore it is best at finding the highest possible or lowest possible output.

The genetic method cannot extrapolate, but is can be useful in AI Trilogy problems where you seek an output with a particular value within the ranges of outputs "seen" during training. In other words, it can interpolate. Do not use it when seeking the highest or lowest possible outputs. On the other hand, the genetic method produces somewhat better contribution factors (relative importance of inputs) to help you determine the usefulness of inputs. The genetic method is also a better choice if there are very few exemplars.

Although both type of nets, neural and genetic, can interpolate, and the neural method can extrapolate, there are limits to how well these things can be accomplished in models. Extrapolation and interpolation can never be accurate if you do not have a rich and comprehensive set of exemplars. Even if you have great exemplars, however, extrapolation can be especially risky when you try to extrapolate too far from the known answers in the exemplars. You are extrapolating into unknown territory.

For example, suppose your output is a quality measurement from 0.0 to 1. Suppose none of your exemplars has a quality above 0.8. You’d like to get a higher quality product. You are much more likely to find the accurate parameters for 0.85 than you are for 1.0. However, after you find some solutions closer to 0.9, and they are verified through actual production, you will have new data with which you can re-train your model. Then you can probably accurately extrapolate to 0.9. Repeating the process, you may later be able to go higher.

A final note: if you are convinced that your data is linear (trends in a straight line relative to each input), you can force the NeuroShell Predictor to build only linear models with the neural method by limiting hidden neurons to zero (see the NeuroShell Predictor Help file.) Linear models extrapolate pretty well if the process being modeled is really linear.


Programming

Once your models are built, they probably won’t change frequently, unless you change the process or the inputs, etc. If the process or the inputs do change, you can simply collect new data and retrain your models. If you have a unique need for frequent retraining, this retraining can be automated inside your own computer programs if you purchase the NeuroShell Engine. Although the Engine is substantially more costly than the AI Trilogy, especially if you plan to distribute your application, Ward Systems Group will credit what you paid for the AI Trilogy towards the purchase of the NeuroShell Engine. Contact Ward Systems Group for details. Also, please read the Programming section under the Optimization Overview.


General Optimization Considerations

Once one or more models are constructed using the neural nets, you will use the Run-Time Server to execute these models within an Excel spreadsheet. A range of cells will contain the inputs, and another cell will be where the network places the output. Whenever input values are placed in the input cells, the spreadsheet will recalculate and give you an output.

It is worth mentioning again that your models don’t have to be neural nets. Some might be formulas or algorithms you already have, assuming you can put them in a spreadsheet (or a computer program – see Programming).

If there are several models made with the Predictor, or the Classifier was used, then there will be several output cells.

Once your model(s) are set up in the spreadsheet, you can "play what-if". What if I use 30% zinc instead of the normal 25% when the temperature is 76 degrees? To find out, put the input values in the spreadsheet and get your answer!

If you have enough time and patience, you could conceivably solve the AI Trilogy problem by trying combinations yourself. If only one or two variables need varying, it might even be practical. Usually, however, the use of GeneHunter will save you a great deal of time and avoid the possibility that a good solution is overlooked.

You tell GeneHunter where the input cells are located. GeneHunter calls the inputs "chromosomes". You also must specify the ranges within which the inputs should be varied. GeneHunter calls these ranges "hard constraints." Be sure not to use ranges very much larger than the ranges of those same inputs varied in the exemplars!

Next you show GeneHunter the output cell(s). GeneHunter calls this the "fitness" function. Another good word for it is the "objective" function. You will tell GeneHunter whether you want it minimized, maximized, or close to a specific value. GeneHunter will then vary the inputs until the specified objective is met.

We have simplified the use of GeneHunter somewhat, because there are more options. Please consult the GeneHunter videos or Help files for specific details on the use of GeneHunter. For advanced topics and understanding, you will want to consult our special web site www.ward.net (this is NOT our sales website!)


Optimization Variations

When finding optimal outputs, you may elect to hold some values constant. No problem, just don’t give those inputs to GeneHunter to vary. Want to greatly limit the ranges? Set the hard constraints any way you want. Suppose you have restrictions, such as the sum of the percentages of zinc and aluminum must be greater than 50%. Just use the "soft constraints" feature of GeneHunter.

The use of several models at one time has already been mentioned. For example, you might want to maximize the quality while minimizing the cost. Use GeneHunter’s multiple fitness function feature. You can even build your own fitness function, where one output is more important than another.

You may want to find several solutions to your problem. A model where several very different combinations of inputs can lead to the same output is called "many to one" in mathematics. The fact that many models are "many to one" is why you can’t just "push the desired output backwards through the neural net to get the proper inputs out the front end." You would get at best an "average" solution of some sort. GeneHunter, however, will usually find different solutions when it is allowed to start with different "random number seeds". Make sure the option to set the random number seed to a particular value is turned off.

GeneHunter has a feature where is will find the 10 best solutions, 100 best solutions, or whatever number you choose. This feature is often useful if you want to see alternate "runner-up" solutions.


Programming Optimization Models

You aren’t required to do your optimization in an Excel spreadsheet. You could make a computer program to do the work if you desire. In a computer program, the optimization will probably be much faster because programs can calculate so much faster than Excel can calculate. Also, you may desire to present the process to the final user in your company with a custom interface. GeneHunter can be programmed in several languages including Visual Basic, Visual Basic for Applications, C, C++, and Delphi. However, the standard GeneHunter license does not permit distribution of such programs outside of your company, but you can negotiate a special license with Ward Systems Group if you desire to distribute or sell your application.


Chemical Reactor Example

Software Used: NeuroShell Predictor and GeneHunter

This example models an imaginary chemical reactor, namely a water titration reactor. The reactor has two tubes feeding in the organic acid solution and the caustic soda solution. Inside the reactor there is a heater. The reactor produces the water solution of a given titration (pH). The reactor operator has two regulators to vary the flow rate of acid and soda, as well as a knob to vary the temperature inside the reactor. It may sound simple, but due to the very complex chemistry between acid and soda when brought together, the resulting solution pH depends in a very non-linear (and still unknown for this type of reactor) manner based upon both fluid flow rates and the temperature.

The task is to build a computer system which would tell to the operator how to set the acid and soda flow rate regulators in order to produce a solution with any given titration pH and for a fixed temperature setting inside the reactor.

Collect Data

The very first step here is to collect the actual data. The operator was asked to run the reactor and to vary the acid flow rate, the soda flow rate, and the reactor temperature in their corresponding maximum ranges and carefully record the resulting pH values for each combination of the process parameters. It turned out that the soda flow rate could be varied in the 45 to 75 range (using arbitrary units), the acid flow rate – in the 3 to 15 range (using arbitrary units again), and the temperature - in the 80 to 160 F range. The resulting pH was measured and was noted to vary considerably between 1 and 13. A total of 100 data records were collected. The data can be found in the pH.csv file in the AI Trilogy\Examples folder.

Build Neural Network Model

The next step is to build a neural network model. The model receives the soda flow rate, the acid flow rate, and the temperature as input variables and produces the pH value as the output. The NeuroShell Predictor does the network training. The resulting trained network is saved as a pH.net file in the same folder.

Run Optimization

The next step is to "reverse" the model using optimization. The operator wants to know how to set the soda and acid flow regulators to achieve pH=7 while maintaining (because of some technological considerations) the constant temperature of 90 F. The Excel worksheet pH.xls does exactly this. Cell E14 contains a formula, which computes the pH for any given process parameters found in cells B14:D14. The formula represents a call to the FireNet function which fires the network pH.net we trained with the process parameters B14:D14. GeneHunter does the process parameter search within the ranges of variation defined above. It varies cells B14 (caustic soda flow rate) in the range 45-75 and C14 (organic acid flow rate) in the range 3-15 to achieve pH=7. Please note that the temperature (cell D14) is kept constant at 90 F and does not vary.

Note 1. The example, though based on real research, is hypothetical. Please do not run your chemical reactors using our settings!

Note 2. Before running the pH.xls worksheet, make sure that the GeneHunter add-in (genehunt.xla) and the NeuroShell Run-Time Fire add-in (nsfire.xla) are setup and selected as add-ins in the Excel Tools/Add-Ins menu.


Cow Feed Optimization Example

Software Used: NeuroShell Predictor and GeneHunter

In this example you are a dairy farmer with a very large herd of cows. You have noticed that different variations of feed cause the cows in your herd to produce different amounts of milk. In fact, you have been feeding your cows a mixture of several ingredients, primarily corn silage, alfalfa hay, high morsel corn, shell corn, soybean meal, oats, barley, regular hay, beef broth, and molasses. Since there are 10 different feeds in the mixture, it is very difficult to see the relationships. You would like to find an optimal mixture in order to maximize milk production.

Collect Data

Fortunately, you have kept records for many months where you have recorded the mixtures fed to the cows and the corresponding milk production resulting from these mixtures. The data is found in the "Cow Feed.csv" file located in the "C:\AI Trilogy\Examples" folder. For each ingredient in the feed, there is a maximum and minimum range of that ingredient which you feel is safe and effective. All values in this imaginary database are in some absolute arbitrary units. These are not percentages, and they do not sum up to 100.

Build Neural Network Model

Next, you run the NeuroShell Predictor to build and train the neural network model. The model has ten inputs (ten ingredients in the feed) and one output (milk production).

In other words, given the mixture amounts, the network outputs the expected milk production when the cows are fed this mixture. If you have a hundred or more examples (mixtures for training) of what happened with various (hopefully widely varying) feed mixtures, you should be able to build a fairly effective predictive net. This net should be able to forecast what milk production to expect with any given new mixtures. The network's success at this will probably depend upon how widely varied your training mixtures were. When running Predictor, do not use the Genetic method because it is not capable of making predictions above those with which it was trained. Use the Neural method instead. An example-trained network is saved as "Cow Feed.net" file in the "C:\AI Trilogy\Examples" folder.

Run Optimization

Now everything is ready to find the optimal feed which will cause the herd of cows to produce the most milk. The optimization part of the problem is setup on the "Cow Feed.xls" Excel worksheet. GeneHunter performs the optimization. Cell L14 contains the fitness function (milk production) to be maximized. The fitness function is a call to the FireNet function that is a part of the NeuroShell Run-Time Server. The FireNet function fires the "Cow Feed.net" with the input data found in cells B14:K14 and returns the predicted milk production value. Cells B14:K14 thus effectively become GeneHunter’s chromosomes. Cells B12:K12 and B13:K13 contains minimum and maximum values of ingredients. GeneHunter varies chromosomes (ingredients) in the defined ranges with the goal to maximize the fitness function (milk production).

Note 1. The example, though based on real research, is hypothetical. Please do not feed your cows any of our feed mixtures!

Note 2. Before running the "Cow Feed.xls" worksheet make sure that the GeneHunter add-in (genehunt.xla) and the NeuroShell Run-Time Fire add-in (nsfire.xla) are setup and selected as add-ins in the Excel Tools/Add-Ins menu.


Ward Systems Group Corporate Headquarters

Ward Systems Group, Inc.

Executive Park West

5 Hillcrest Drive

Frederick, MD 21703

USA

Phone: (301) 662 7950

Fax: (301) 663 9920

E-mail: sales@wardsystems.com

Sales: http://www.wardsystems.com


Ward Systems Group Sales Department

Ward Systems Group Inc. sales hours are: 8:30 a.m. to 5:30 p.m. Eastern time.

Telephone: (301) 662 7950

Fax: (301) 663 9920

Email: sales@wardsystems.com

Web: http://www.wardsystems.com


Ward Systems Group Tech Support

If you have a question about how the AI Trilogy works, the best way to get an immediate answer is to check the on-line Help file.

We are dedicated to providing quality technical support. In fact, our technical support staff has acquired such an outstanding reputation that we frequently get help requests from people who are not entitled to this service. To keep these people from getting in the way of providing quality service to legitimate users of our products, we will always ask for your name and serial number. Please assist us by understanding that this helps you get the fastest support. The serial number is on the back of your CD case.

Here are the steps you should take to get help:

a. Check the Help file. It is quite extensive, and we have found that it covers 90% of all questions we are asked about this product.

b. Check our tech support web page (for users only). On this page, we will be posting reports of bugs, workarounds, fixes, frequently asked questions, tips, notice of new releases, and new examples. You can also look for help on the Discussion Forum located on the web page. You will need your serial number to log into the product area for this product. The web page address is www.ward.net.

This web page will always be changing, so check it frequently.

c. E-mail or fax us your questions. Be sure to include your name and serial number. We try to answer all questions within 24 hours (Monday through Friday except US holidays and days of certain company events). However, you must carefully describe the problem you are having. In particular, copy the exact text of any error message, and exactly what you were doing when you received it. Most people assume we are familiar with the error, but many of them come from Windows. Our e-mail address is: support@wardsystems.com.

Our fax number is (301) 663-9920.

New or alternate addresses may be posted from time to time on our tech support web page (see b. above).

d. Call us on the phone. We will ask for your name and serial number. Please have it ready! Our phone tech support service is open from 9:00 am to 12 noon, and from 1:00 PM to 5:00 PM Eastern Time Monday through Friday except US holidays and days of certain company events. The phone number is (301) 662-7950. The busiest times to call are between 11:00 am and 3:00 PM Eastern Time. The busiest days are Monday and Friday. In the event that you call us at a busy time when all of our tech support personnel are assisting others, you may hold or call back at a less busy time.

e. You may pay for personal consulting in our office. This service is by appointment only and costs $200 per hour in quarter hour increments.

f. You may pay for consulting or classes in your office. This service is arranged on a case-by-case basis.


NeuroShell Engine

The NeuroShell Engine is an Active X control that can be loaded into your Visual Basic, C++, or Delphi programs to bring the neural network power of the NeuroShell Predictor and Classifier (including training) to your programs.

Background
The NeuroShell Engine is a product that most of our users will never need, because our other programs allow you to call your networks from computer programs you write. Such computer programs can be distributed throughout your company and to your customers. If you purchase the NeuroShell Predictor or NeuroShell Classifier, you simply have to purchase the NeuroShell Run-Time Server in order to place calls to your nets into your Windows programs written in Visual Basic, C++, Delphi, and other languages. If you purchase NeuroShell 2, it includes a runtime server so you can call your programs from the aforementioned languages in Windows, and you can also generate source code so your nets can be compiled on mainframes or Unix workstations.

So why do we sell the NeuroShell Engine? In case you want to embed the TRAINING portion of the process into your Windows programs. Let us explain further.

Use of neural nets involves two stages:

Stage 1, TRAINING. In this stage, the historic or sample observations are collected to build the neural model. The process is called training, where the net "learns" the sample observations. The end result is a model that can be used in stage 2.

Stage 2, FIRING (executing). In this stage, the model built in stage 1 is called (or "fired") to produce answers for new input data, data that was normally not used in stage 1 to build the model. This is the production environment where you utilize the model to make predictions, classifications, or other decisions.

Of course, our programs (NeuroShell Predictor, NeuroShell Classifier and NeuroShell 2) provide both stage 1 and stage 2 in them in graphical user interfaces. However, only stage 2 can be controlled from within your own computer programs written by your own programmers. (NeuroShell 2 includes this capability; NeuroShell Predictor and Classifier require the Run-Time Server.)

Stage 1, in all of our other products, is accomplished only from a graphical user interface that is built into our products. That's where the NeuroShell Engine comes in. The Engine lets you call stage 1 (training) from your computer programs. In essence, it allows you to build your own neural network products for distribution! You can build custom designed neural network product interfaces too, by dropping our engine into your chassis.

What is the NeuroShell Engine
The NeuroShell Engine is only for the most serious neural network users, and only those who are programmers or have programmers on staff. The NeuroShell Engine contains the neural and genetic training methods that we have used ourselves in the NeuroShell Predictor, the NeuroShell Classifier, and the NeuroShell Trader. They are available to be integrated into your own computer programs for both training and firing neural networks. Basically there are four algorithms:

1. The neural method (Turboprop 2) used in the Predictor and Trader
2. The genetic method (Advanced GRNN) used in the Predictor
3. The neural method (Turboprop 2) used in the Classifier
4. The genetic method (Advanced PNN) used in the Classifier

Licensing and Pricing
Please don't look for this engine to be inexpensive like most of our products have always been. We're letting you have the crème de la crème of neural network modeling, the technological heart of our most powerful products. If you aren't a very serious user, you will not want to pay the price for serious networks. We aren't talking about the classic "backprop" neural network paradigms that every college student learns. If you are looking for an inexpensive solution, we recommend backprop, which is not in the NeuroShell Engine.

There will be two different types of licenses:

1. Internal use only license. With this license, you can use the NeuroShell Engine in all programs which you intend to distribute only within your company or government agency. There will be no restrictions of what type of program you may build, but the license will not allow it to be used by anyone but your own employees. All of your programmers in one physical location may use the software.

2. External use license. With this license, which will be the more expensive of the two, you may use the NeuroShell Engine in a specified program which you intend to sell or otherwise distribute outside of your company or government agency. We must approve the program to be sure that it does not directly compete with any of our products. Once your application program is approved, you may embed our technology and sell or distribute the program with no royalty payments. All of your programmers working on your application may use the software.