8 a] Explain the variants of the CNN model.
Multi-Channel Convolution:
- In a CNN, convolution is typically applied using multi-channel input, such as RGB images with three channels (red, green, and blue). The convolution operation is extended to multi-dimensional tensors (3D for each input and output layer).
- Each channel can have different filters, and these filters can interact across multiple spatial locations, unlike standard convolutions that typically use single-channel inputs.
Stride Convolution:
- Stride refers to the number of pixels the kernel moves during the convolution operation. A stride of 1 means the kernel moves one pixel at a time, while a stride greater than 1 reduces the size of the output by sampling fewer locations.
- Stride convolution is a downsampling operation that speeds up computation by reducing the resolution of the feature maps at the cost of less fine-grained features.
Zero Padding:
- Zero padding is the practice of adding zeros around the borders of the input image to control the output size. Padding can prevent rapid shrinking of the image size during multiple layers of convolution and enable networks to capture more edge features.
- There are three padding strategies:
- Valid Convolution: No padding is applied, which causes the output size to shrink after each convolution layer.
- Same Convolution: Padding is added to maintain the output size equal to the input size.
- Full Convolution: Padding is added to ensure every pixel is visited by the kernel multiple times, resulting in an output size that is larger than the input.
Locally Connected Layers:
- In a locally connected layer, weights are not shared across spatial locations. Each location in the output has its own set of weights. This is sometimes called “unshared convolution.”
- This approach can be useful when you want each feature to be a function of a small region but not necessarily the same feature across the entire spatial area (e.g., detecting faces but looking for different features in different parts of the image).
Tiled Convolution:
- Tiled convolution strikes a balance between convolution and locally connected layers. Instead of learning a separate set of weights at every spatial location, a set of kernels is learned, which are cycled through spatial locations.
- It helps to reduce memory consumption while allowing each location to have different filters like in locally connected layers. The weight-sharing pattern cycles through a set of kernels as the convolution moves across the input.
Backpropagation in Convolutional Layers:
- To train convolutional networks, it’s necessary to compute gradients using backpropagation. Convolution, especially when using strides, requires additional steps to compute the gradient with respect to the kernel and the input image.
- The forward and backward passes can be described as matrix multiplications, and specialized algorithms such as transpose convolutions are used to backpropagate errors through the layers.
Bias Sharing:
- In most convolutional layers, the bias term is typically shared across all spatial locations within the same feature map. However, in locally connected and tiled convolution layers, it might be more natural to use different biases or share them with the same tiling pattern as the kernels.