A neural network is a computational model based on the information-processing mechanisms of biological neural networks found in the human brain. Its functions include pattern recognition, data classification, and example-based prediction making. Deep learning is a subset of machine learning, and one of its main components is neural networks.
Fundamental Components and Structure
Source: Neural Networks
Neurons
A neural network’s fundamental computational unit is a neuron, sometimes referred to as a node or unit. Every neuron takes in information, processes it, and then releases an output. Usually, the processing entails:
Input Weights (𝑤): Every input has a corresponding weight that modifies how much of an impact it has on the output of the neuron.
Weighted Sum: The weighted sum of the inputs is calculated by the neuron.
Activation Function (𝜙): The activation function (𝜙) gives the model non-linearity, which enables it to pick up intricate patterns.
Layers
The structure of neurons is layered:
Input Layer: The initial layer that receives input data is known as the input layer. An aspect of the input data is represented by each neuron in this layer.
Hidden Layers: Intermediate layers that handle the inputs are known as hidden layers. The network picks up pattern recognition skills in these layers, which can have one or more of them.
Output Layer: The output layer is the last layer that generates the network’s output. The type of challenge (e.g., number of classes for classification) determines the number of neurons in the output layer.
Weights and Biases
The network’s parameters are weights and biases. They become available during training:
Weights (𝑤): Modify how much each input affects the output of the neuron.
Biases (𝑏): Offer an extra degree of freedom by allowing adjustments to be made to the output in addition to the weighted sum of the inputs.
Method of Learning
Source: Learning Process
Forward Propagation
The input data travels over the network layer by layer during forward propagation. After applying an activation function and computing the weighted sum of its inputs, each neuron sends the output to the layer below. Until the output layer generates the final output, this procedure is repeated.
Function of Loss
The difference between the intended output and the predicted output is measured by the loss function. It directs the learning process and evaluates the inaccuracy or gap.
Backpropagation
The technique of backpropagation involves adjusting the weights and biases of the network to minimize the loss function. It includes:
Compute Gradients: Using the chain rule, determine the gradient of the loss function concerning each weight and bias.
Modify the Biases and Weights: Gradient descent is a common optimization approach used to adjust the weights and biases in a direction that minimizes the loss.
The actions to be taken are:
Forward Pass: Determine the loss and the output.
Backward Pass: Determine the gradients of the loss for each weight and bias in the backward pass.
Weight Update: Utilising the gradients, update the weights and biases.
Optimization Algorithms
Some algorithms for optimization consist of:
Stochastic Gradient Descent (SGD): Using the gradient of a single data point, stochastic gradient descent (SGD) updates weights.
Mini-batch Gradient Descent: This method computes and updates the weights using a small batch of data points.
Adam: Combining the benefits of RMSProp and AdaGrad, Adam modifies the learning rate in an adaptable manner.
Artificial Neural Network Types
Feedforward Neural Networks (FNN): The most basic kind of neural network, in which connections don’t repeat. From input to output, information flows in a single direction.
Convolutional Neural Networks (CNN): Convolutional neural networks (CNN) are primarily employed in tasks related to image processing. Convolutional layers are a tool used by CNNs to automatically and adaptively extract spatial feature hierarchies from input images. Important elements consist of:
Source: Types
Convolutional Layers: Use convolutional processes to identify characteristics. Lower the spatial dimensions (downsampling) in the pooling layers.
Fully Connected Layers: Classify data according to identified attributes.
Recurrent Neural Networks (RNN): RNNs, or recurrent neural networks, are made to process sequential input. RNNs’ cycle-forming connections enable the persistence of information. For applications like natural language processing and time series prediction, they are especially helpful.
RNNs with Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU) variants are more efficient for longer sequences by resolving the vanishing gradient issue.
Artificial neural networks are flexible and effective methods for representing complicated patterns in data. They are made up of layers of networked neurons, each of which can carry out basic computations. Neural networks employ a three-step procedure that involves forward propagation, loss computation, and backpropagation to acquire the ability to accurately anticipate data. It is essential to understand neural network components, learning algorithms, and the underlying mathematical principles to apply neural networks to a variety of tasks in an efficient manner.