Section 1.1: RoI Pooling Explained

Let’s clarify the differences between RoI Pooling and RoI Align. Both techniques aim to enhance the retention of information as data transitions from one layer to the next. Before introducing RoI Warping, we should first examine these two methods.

After extracting features from a convolutional neural network, we generate numerous feature maps, preparing for region proposal generation. For instance, when utilizing VGG16 as the backbone and an initial scale factor of 32, the generated region proposals correspond to coordinates in the original image. When these coordinates are transferred to the feature map, they undergo quantization, resulting in some loss of information.

Imagine this process: transitioning our region proposal from the image to the feature map inevitably incurs slight information loss and potentially introduces unnecessary data due to integer rounding.

The green area illustrates the quantization effect, while the red area signifies the data loss. Initially, the region proposal measures 4x6 on the feature map. The next step further quantizes this to an arbitrary dimension, such as 3x3 (it could also be a 7x7 pooling layer), before these features are directed into the final layer. The separation of the RoI layer into bins precedes the max-pooling operation.

Upon quantization, the RoI layer simplifies; we only need to extract the maximum value from each bin, resulting in a 3x3 matrix. Due to the fixed input size of 224x224 for VGG16, our output appears as 3x3x224 matrices. Thus, the essence of RoI Pooling is that it inevitably incurs some information loss during the quantization stages.

Section 1.2: RoI Align in Depth

In contrast to RoI Pooling, which applies two rounds of quantization—leading to significant information loss—RoI Align aims to mitigate this issue by avoiding quantization during both mapping and pooling stages. A fundamental understanding of bilinear interpolation is helpful here.

In this method, we retain the original coordinates without quantization. For the pooling stage, using a 3x3 pooling layer (which can be adjusted to a 7x7 layer), we partition the RoI feature map into 3x3 grids. The key distinction from RoI Pooling is that RoI Align samples four data points from each grid box to facilitate bilinear interpolation.

To determine these sampling points, you can apply the following formulas:

X = X_coord_box + (width / pooling_layer_size) * sampling_point_id

Y = Y_coord_box + (height / pooling_layer_size) * sampling_point_id

With the four data points identified, we can implement bilinear interpolation. By gathering the maximum values from the nine grid boxes, we produce our final 3x3 matrix.

As you will observe, RoI Align significantly reduces information loss by fully leveraging feature maps for data pooling, albeit at the cost of increased computational demands.