Pooling & Feature Maps

Pooling replaces each local feature-map window with a summary: its maximum or mean. With a stride greater than one, it reduces spatial resolution while keeping channels separate. This saves stored values but discards spatial detail; moving a response across a window boundary can still change the output.

Think of a feature map as a sheet of measurements. Replacing each small block with one number makes the sheet shorter. It also limits what you can ask afterward: the maximum tells you how strong the strongest response was, but the output value alone cannot tell you where inside the block it occurred.

Use the smaller maps when later calculations need regional summaries. If you still need exact response coordinates, keep the feature maps: two different positions can produce the same pooled output. The example below puts that tradeoff into numbers: 72 values become 18, and a one-pixel shift becomes impossible to recover from the output alone.

How Pooling Works on Feature Maps

A feature map records one filter's response at different spatial positions. Two filters applied to the same image produce two maps, even if their responses happen to look similar. Those maps are separate channels. A channel here identifies a filter, not a predicted class or necessarily a red, green, or blue image component.

This example starts with an 8×8 image containing one bright pixel of intensity 8 at zero-based position (3,2). Every other pixel is zero. Two fixed 3×3 filters measure opposite horizontal brightness differences:

        right − left             left − right
          0  0  0                  0  0  0
         −1  0  1                  1  0 −1
          0  0  0                  0  0  0

Each filter slides over valid 3×3 patches, without padding or a kernel flip. ReLU replaces negative responses with zero. The result is two 6×6 feature maps: channel 0 has an 8 at (2,0), and channel 1 has an 8 at (2,2). All their other entries are zero. The filters are deliberately simple and untrained; their responses are calculated, not invented activations from a classifier.

Pooling now operates within each channel. With a k×k window starting at (r,c), max pooling takes the largest of its k² entries. Average pooling sums all k² entries and divides by k², including any zeros. For the window below, the results differ:

8  0        max = 8
0  0        average = (8 + 0 + 0 + 0) / 4 = 2

Max emphasizes the strongest response. Average measures the mean response across the region, so a single isolated peak contributes less as the window grows. Neither choice knows whether that peak came from a useful feature or an unwanted disturbance. Choose the summary that suits the representation you need.

The stride is the distance between successive window starts. Here it equals the window size: 2×2 windows start two entries apart and do not overlap. A 6×6 map therefore becomes 3×3. Apply that operation to both channels and the complete shape, written height × width × channels, changes from 6×6×2 to 3×3×2. There are 18 stored values instead of 72. The two channels remain separate.

Pooling & Feature Maps Step by Step

The default uses max pooling, window 2×2, stride 2, and no padding.

1. Calculate one window

Start with channel 0's feature rows 2–3 and columns 0–1. This is a teaching probe chosen to include the response of 8. Its four entries are [[8,0],[0,0]], so max pooling produces 8. Write that value to pooled position (1,0). The source feature map remains unchanged; pooling produces a new map.

2. Complete the maps separately

The other eight windows of channel 0 contain only zeros and each produces zero. Channel 1 follows the same rule. Its response at feature position (2,2) belongs to the window starting at (2,2), so it writes 8 to pooled position (1,1).

Channel 0 pooled       Channel 1 pooled
    0  0  0                0  0  0
    8  0  0                0  8  0
    0  0  0                0  0  0

Return to the first window, [[8,0],[0,0]], before changing the aggregation rule. Its maximum is 8; its mean is (8 + 0 + 0 + 0) / 4 = 2. The contrast keeps these operands fixed and shows both summaries. The selected output map still uses Max. If you select Average, the two nonzero pooled values become 2; the windows, output positions, and channel count stay the same.

3. Move one pixel right

Move the bright input pixel from (3,2) to (3,3) and recompute both filters. Channel 0's response moves from feature (2,0) to (2,1); channel 1's moves from (2,2) to (2,3).

Both responses stayed within their original 2×2 pooling windows. Consequently, both pooled maps remain exactly as printed above. The full feature maps changed, but every pooled value stayed the same.

That equality proves information loss. If you were handed only the pooled tensor, you could not distinguish these two different feature-map tensors. Pooling's output does not retain their exact within-window positions.

4. Cross a window boundary

Move the bright pixel two pixels right of the original, to (3,4). The feature responses now occupy (2,2) in channel 0 and (2,4) in channel 1. Each has entered the next pooling window:

Channel 0 pooled       Channel 1 pooled
    0  0  0                0  0  0
    0  8  0                0  0  8
    0  0  0                0  0  0

Four scalar entries differ from the original pooled tensor: each channel loses its old 8 and gains an 8 at a new position. One response moved, but two stored cells changed: 8 → 0 at the old position and 0 → 8 at the new one. Across both channels, that is 2 + 2 = 4 changes. Pooling hid the first translation for this input, but it did not hide the second. All three source positions remain inside valid filter support, so image clipping does not explain the change.

Window Size, Spatial Detail, and Pooling Cost

For an input of height H and width W, a square window k, stride s, and no padding or dilation, the output dimensions are:

H_out = floor((H − k) / s) + 1
W_out = floor((W − k) / s) + 1
C_out = C_in

This lesson offers k = s = 1, 2, or 3. Each divides the 6×6 input map exactly; arbitrary sizes can leave a border outside the last valid window. These formulas follow the PyTorch pooling definitions; the average-pooling definition uses the same spatial indexing with an arithmetic mean.

A larger window does not guarantee an unchanged output for every smaller shift. At size 3, channel 1's response moves from column 2 to column 3 after a one-pixel translation and crosses a window boundary immediately. Its original and new pooled positions differ. This is why window alignment belongs in any claim about translation tolerance.

The direct pooling calculation has time complexity O(C × H_out × W_out × k²) and stores O(C × H_out × W_out) output values. For the non-overlapping windows that exactly cover these maps, the time simplifies to O(CHW): each input entry participates in one reduction. Fewer output values can reduce work in later layers, but a 75% reduction in stored elements is not a measurement of information lost or a promise of a 75% runtime improvement.

Pooling Edge Cases

• A 1×1 window: max and average both return their only input. No spatial detail is discarded and no downsampling occurs with stride 1.
• All-zero windows or tied maxima: the maximum is well defined even when several entries attain it. The value does not identify a unique source coordinate. The explanation outlines all tied maxima.
• An isolated peak under average pooling: enlarging the window changes its contribution from 8 to 2 to 8/9 in these settings. The fixed 0–8 color scale keeps that attenuation visible; the display does not rescale each result to look equally strong.
• Padding or overlapping windows: both are outside this example. Their boundary and counting rules matter; a formula derived for non-overlapping, unpadded windows should not silently cover them.

Common Pooling Mistakes

• Calling pooling a reversible resize. The default original and one-pixel-shifted feature maps produce identical pooled tensors. An inverse receiving only that tensor cannot tell which map produced it.
• Pooling across channels by accident. Spatial pooling preserves the filter identity of each map. Combining channels asks a different question and loses that separation.
• Calling the result a prediction. These values are local filter-response summaries, not class probabilities or object labels.
• Assuming max pooling removes noise. A large unwanted response can be the maximum too. The operation has no knowledge of the response's cause.
• Claiming full translation invariance. Whether these outputs change depends on which window receives each response. The boundary comparison is part of the explanation, not an exception to ignore.

A Note on Simplification

This is an explainer built from a synthetic grayscale image, two fixed contrast filters, ReLU, and non-overlapping pooling. It is not a substitute for formal study of convolutional networks. Training, learned filter banks, normalization, padding variants, gradients, and downstream classification are omitted. Pooling is one way to downsample; a CNN need not contain a pooling layer. Continue with CNN to change a filter itself, or Image Classification to see how a real classifier's predictions depend on the image crop it receives.