Let us start with a logistic regression. Recall that for a logistic regression model we represent our data as point coordinates. The model allows us to split all the points into 2 classes. Visually the model divides 2 classes of data records by a line (or a plane, or a hyperplane if we have higher dimensions) .
Numerically a logistic regression yields values from 0 to 1 for each data point. For every record we compute a probability for it to belong to one class or another and we can consider the process as making an evaluation. A value close to 1 means one class, while close to 0 means another. In the process we load data and calculate our evaluation by a formula.
For example we may have the following assignment: to compute if we have enough goods in warehouses to last for a week of sales. This is quite a common problem, and say inventory clerks report amounts of each item to a manager. The manager collects information, processes it and makes an evaluation: do we have enough of every product, "Yes" or "No".
Here we can use a logistic regression model which produces probabilities for 2 classes: "Yes" and "No". If you feel that our manager should supply a bit more information with it, for example, how much should be restocked if any, then in Neural Networks it could passed, too.
Usually one evaluation is not the only required indicator for a chain of stores to operate. In addition we need to know, for example, if our warehouses are full to optimal capacity (75% - 85% or something like this). Therefore we need to estimate this as well. In our organization we will have another manager gathering data and assessing if we have enough reserve storage space.
Then let us say that these people should report to their supervisor who will make next evaluation about store preparedness level:
So we get a whole hierarchy of managers, submitting their assessments and at the end the company CEO obtains a summary report. The CEO looks at the whole picture, issues adjusted guidelines for further work and sends them back to middle management. We can compare it with a neural network structure:
Now we can observe a lot of in common with a corporation chain of command. As we see middle managers are corporation hidden layers which do the balk of the job, as it happens in real life. They got reports from their subordinates and guidelines form their bosses, and they optimize their work based on the provided information. In neural networks we have the similar information flow and processing which is analogous to forward propagation (from lowest workers to CEO) and backward propagation (instructions from CEO to middle management, adapted for each lower manager by her/his boss).
I discovered that a lot
of people express a desire for the NN results to be more explainable by
its inner functioning. I believe that my analogy is helpful for this,
too. As
we know, a company success is defined by its top management. They
select a company structure and define an inner company policy, which middle management
adapts and directs further by usual means. We know that an industry
leader could originate due to a chance, a lucky initial idea. But its
following performance is a learned behavior. People use experience of
other company managers combining it with their own trial & error
process.
Now let us look at what the values in last 2 columns.
