Neural Networks

Here we will define and explain Neural Networks from scratch. We will also attempt to create a NN using Python code.

Neural Networks are structures that try to emulate the neurons in the brains of living biological beings. They are at the core of deep learning algorithms and are thus integral to Artificial Intelligence.

In their simplest form, NN are analogous to functions in mathematics; in fact, that is exactly what they are. The basic function, \(f(x) = y\) , takes as input the value for \(x\) and gives an output of \(y\) . Likewise, neural networks consist of inputs and outputs.

NN's consist of layers of nodes. There is the input layer, the hidden layer and the ouput layer. Data is fed into the input layer of nodes and based on the state of the data and of the condition of the nodes, a decision is made to arrive at some outcome. Below we will explain how these nodes function and operate.

Perceptron

One of the simplest forms of NN is known as a perceptron. A perceptron is a single node consisting of input data, weights, a bias and an output. Input can be thought of as certain conditions or decisions to be made, weights can be used to determine the importance of those decisions and a bias can be used to determine whether or not the node will fire. The formula for the perceptron looks like this:

\[\sum_{i=1}^{m}{w_i}{x_i} + b = w_1x_1 + w_2x_2 + w_mx_m + b\]

The above formula states that multiplying each input \(x_i\) by its weight \(w_i\) and adding them together along with the bias \(b\) will give us a value.