Deep Learning is a subfield of Machine Learning that involves the use of neural networks, a process similar to how the neurons in the human brain function.
Neural networks are composed of interconnected artificial neurons. Some neurons are input neurons that provide weighted inputs to the next layer, called the hidden layer, which has neurons that are functions. Some neurons are output neurons. The ‘deep’ refers to the depth of these neuron layers
Types of Neural Networks
A feedforward neural network is one that has all inputs only moving forward i.e. no loops.
Some networks have neurons that link back to the input/hidden layer neurons, resulting a feedback loop. This is known as back propagation.
Perceptron
Neurons used in the hidden layer of a neural network are named for the activation function that they use. The artificial neuron known as a perceptron accepts inputs, and outputs yes/no or 0/1, making it a binary classifiers
After training, a perceptron realises that inputs can either fire or not fire the neuron. That means the data set is classified as linearly separable. So the line dividing the two, whether is the decision line which separates the labelling of the neural network.
Sigmoid Neurons
Sigmoid neurons are similar to perceptrons, in that they use inputs with discrete weighted importance, and a bias value. However, they do not give a binary output as perceptrons do (using the step function). Instead, the output from a sigmoid neuron is a real number between 0 and 1.
They use the sigmoid function () to evaluate the output.
Training Neural Networks
Advantages & Disadvantages
Pros:
- Makes use of all the training data, no data is wasted. Therefore more accurate, for a well-labelled dataset.
- Accuracy increases as more data is fed
- Useful when no known algorithms exist for solving a problem. The neural network can just be fed the inputs, and then the outputs it produces can be verified.
Cons:
- Requires significantly more data than SVMs
- Very opaque, hard to see ‘train of thought’
- Much slower than SVMs
Resources
3Blue1Brown’s video on neural networks
How to train a network