Skip to main content
 首页 » 编程设计

python之CNN : Forwarding batch of images之Stuck when using multiple kernels

2024年11月24日9mfrbuaa

我把它放在第一位,我没有使用任何框架,我像上周一样开始学习卷积层。我了解前向传播的基础知识,甚至还没有回过头来,但这超出了范围。这是我的困惑:

假设我有 4 张图像,有 3 个大小为 4x4 的 channel :4x3x4x4 我有 3 个大小为 3x3 的 channel 的内核:K*3x3x3

我正在尝试计算所有 4 个图像的卷积,但我总是迷失在维度中。这是我尝试过的:

import numpy as np 
 
 
w = np.array( 
    [ 
        # Img: 1, 4x4 image with 3 channels 
        [ 
            [ 
                [1, 1, 1, 1], 
                [1, 1, 1, 1], 
                [1, 1, 1, 1], 
                [1, 1, 1, 1] 
            ], 
            [ 
                [0, 0, 0, 0], 
                [0, 0, 0, 0], 
                [0, 0, 0, 0], 
                [0, 0, 0, 0] 
            ], 
            [ 
                [2, 2, 2, 2], 
                [2, 2, 2, 2], 
                [2, 2, 2, 2], 
                [2, 2, 2, 2] 
            ] 
        ], 
        # Img: 2, 4x4 image with 3 channels 
        [ 
            [ 
                [0, 0, 0, 0], 
                [0, 0, 0, 0], 
                [0, 0, 0, 0], 
                [0, 0, 0, 0] 
            ], 
            [ 
                [1, 1, 1, 1], 
                [1, 1, 1, 1], 
                [1, 1, 1, 1], 
                [1, 1, 1, 1] 
            ], 
            [ 
                [2, 2, 2, 2], 
                [2, 2, 2, 2], 
                [2, 2, 2, 2], 
                [2, 2, 2, 2] 
            ] 
        ], 
        # Img: 3, 4x4 image with 3 channels 
        [ 
            [ 
                [2, 2, 2, 2], 
                [2, 2, 2, 2], 
                [2, 2, 2, 2], 
                [2, 2, 2, 2] 
            ], 
            [ 
                [0, 0, 0, 0], 
                [0, 0, 0, 0], 
                [0, 0, 0, 0], 
                [0, 0, 0, 0] 
            ], 
            [ 
                [1, 1, 1, 1], 
                [1, 1, 1, 1], 
                [1, 1, 1, 1], 
                [1, 1, 1, 1] 
            ] 
        ], 
        # Img: 4, 4x4 image with 3 channels 
        [ 
            [ 
                [2, 2, 2, 2], 
                [2, 2, 2, 2], 
                [2, 2, 2, 2], 
                [2, 2, 2, 2] 
            ], 
            [ 
                [1, 1, 1, 1], 
                [1, 1, 1, 1], 
                [1, 1, 1, 1], 
                [1, 1, 1, 1] 
            ], 
            [ 
                [0, 0, 0, 0], 
                [0, 0, 0, 0], 
                [0, 0, 0, 0], 
                [0, 0, 0, 0] 
            ] 
        ] 
    ] 
) 
 
f = np.array( 
    [ 
        # Filter: 1, 3x3 filter for 3 channels -> All 1s 
        [ 
            [ 
                [1, 1, 1], 
                [1, 1, 1], 
                [1, 1, 1] 
            ], 
            [ 
                [1, 1, 1], 
                [1, 1, 1], 
                [1, 1, 1] 
            ], 
            [ 
                [1, 1, 1], 
                [1, 1, 1], 
                [1, 1, 1] 
            ] 
        ], 
        # Filter: 2, 3x3 filter for 3 channels -> All 2s 
        [ 
            [ 
                [2, 2, 2], 
                [2, 2, 2], 
                [2, 2, 2] 
            ], 
            [ 
                [2, 2, 2], 
                [2, 2, 2], 
                [2, 2, 2] 
            ], 
            [ 
                [2, 2, 2], 
                [2, 2, 2], 
                [2, 2, 2] 
            ] 
        ] 
    ] 
) 
 
hori_dimension = (w.shape[3] - f.shape[3]) // 1 + 1 
vert_dimension = (w.shape[2] - f.shape[2]) // 1 + 1 
r = np.zeros(shape=(w.shape[0], f.shape[0], vert_dimension, hori_dimension)) 
for i in range(vert_dimension): 
    for j in range(hori_dimension): 
        r[:, :, i, j] += np.sum(w[:, :, i:i+3, j:j+3] * f, axis=(1, 2, 3)) 
print(r) 

这不起作用,当我有 N 个图像的 K 个内核时,这部分我有问题。

但是,如果我只有一个内核,那么我可以将我的操作定义为(这完美地工作):

hori_dimension = (w.shape[3] - f.shape[3]) // 1 + 1 
vert_dimension = (w.shape[2] - f.shape[2]) // 1 + 1 
r = np.zeros(shape=(w.shape[0], vert_dimension, hori_dimension)) 
for i in range(vert_dimension): 
    for j in range(hori_dimension): 
        r[:, i, j] += np.sum(w[:, :, i:i+3, j:j+3] * f, axis=(1, 2, 3)) 
print(r) 

这导致每张图片的 2x2 特征:

[[[27. 27.] 
  [27. 27.]] 
 
 [[27. 27.] 
  [27. 27.]] 
 
 [[27. 27.] 
  [27. 27.]] 
 
 [[27. 27.] 
  [27. 27.]]] 

这似乎是对的,我有 1 个内核,所以我将为每张图像提供 1 个特征图,从而产生 4 个 2 维特征图。在上面我期望,另一个维度。每个内核有 4 个特征映射,但我无法弄清楚这个。

@更新:

这看起来像是在做这项工作:

p = 0 
s = 1 
number_of_input_images, number_of_image_channels, height_of_image, width_of_image = w.shape 
number_of_kernels, number_of_kernel_channels, height_of_kernel, width_of_kernel = f.shape 
assert(number_of_image_channels == number_of_kernel_channels) 
 
width_of_features = (width_of_image - width_of_kernel + 2*p) // s + 1 
height_of_features = (height_of_image - height_of_kernel + 2*p) // s + 1 
feature_maps = np.zeros(shape=(number_of_input_images, number_of_kernels, height_of_features, width_of_features)) 
 
for k in range(f.shape[0]): 
    for i in range(height_of_features): 
        for j in range(width_of_features): 
            feature_maps[:, k, i, j] += np.sum(w[:, :, i:i+3, j:j+3] * f[k], axis=(1, 2, 3)) 
 
print(feature_maps) 

它产生以下特征图:

[ 
    # pic1 
    [ 
        # kernel1 
        [ 
            [27. 27.] 
            [27. 27.] 
        ] 
        # kernel2 
        [ 
            [54. 54.] 
            [54. 54.] 
        ] 
    ] 
    # pic2 
    [ 
        #kernel1 
        [ 
            [27. 27.] 
            [27. 27.] 
        ] 
        #kernel2 
        [ 
            [54. 54.] 
            [54. 54.] 
        ] 
    ] 
    #pic3 
    [ 
        #kernel1 
        [ 
            [27. 27.] 
            [27. 27.] 
        ] 
        #kernel2 
        [ 
            [54. 54.] 
            [54. 54.] 
        ] 
    ] 
    #pic4 
    [ 
        #kernel1 
        [ 
            [27. 27.] 
            [27. 27.] 
        ] 
        #kerbel2 
        [ 
            [54. 54.] 
            [54. 54.] 
        ] 
    ] 
] 

有更好的方法吗?这甚至是正确的吗?对我来说似乎很好。有一张图片和多个内核,卷积的结果将是每个内核的特征图放在另一个内核之后,对吧?因此,具有 K 个内核,其中特征图具有 NN 维,卷积层的输出变为 KN*N。那样的话,我猜上面的似乎是正确的?正如我所说,我真的搞砸了这 N 个维度......

@更新:

我最终得到了以下用于有效(前向)/完整(反向传播)卷积的代码:

def convolve(sources: np.ndarray, 
             kernels: np.ndarray, 
             mode: str = 'valid', 
             padding: typing.Tuple[int] = (0, 0), 
             stride: int = 1): 
    number_of_input_images, number_of_image_channels, height_of_image, width_of_image = sources.shape 
    number_of_kernels, number_of_kernel_channels, height_of_kernel, width_of_kernel = kernels.shape 
    assert(number_of_image_channels == number_of_kernel_channels) 
 
    if mode == 'full': 
        padding = (height_of_kernel, width_of_kernel) 
 
    if padding: 
        sources = np.pad(sources, 
                         ((0, 0), (0, 0), (padding[0], padding[0]), (padding[1], padding[1])), 
                         mode='constant', constant_values=0) 
 
    kernels = np.rot90(kernels, k=2, axes=(2, 3)) 
 
    width_of_features = (width_of_image - width_of_kernel + 2*padding[1]) // stride + 1 
    height_of_features = (height_of_image - height_of_kernel + 2*padding[0]) // stride + 1 
    feature_maps = np.zeros(shape=(number_of_input_images, number_of_kernels, height_of_features, width_of_features)) 
 
    for k in range(f.shape[0]): 
        for i in range(height_of_features): 
            for j in range(width_of_features): 
                feature_maps[:, k, i, j] = np.einsum('ncij,cij', sources[:, :, i:i+3, j:j+3],  kernels[k]) 
 
    return feature_maps 

如有任何反馈,我们将不胜感激。我读到内核在进行卷积时必须旋转,所以我旋转了两次 90 度,还可以使用自定义填充,并且为了获得完整的卷积,我用内核的大小 -1 进行填充,以便所有周围的元素为零,我没有得到索引错误。

请您参考如下方法:

让我们一次看一张图像和一个内核。如果您的图像大小为 wxh 并且您的内核大小为 f*f 并且如果您一次跨一个像素并且如果您使用 填充 iamge code>p 像素然后在 1 个图像与 1 个内核卷积之后将产生大小为 (w-f+2*p)/s + 1, (h-f+2*p )/s +1)。在您的情况下,w=h=4、f=3、s=1 和 p=0。

  1. 首先,您从图像中获取一个f*f 补丁。由于您有 3 个 channel ,因此每个补丁都是 3 个 channel
  2. 每个 channel 的内核与补丁中的相应 channel 相乘(逐元素相乘)
  3. 最后,所有 channel 中的所有数字相加产生一个数字。

图示

多个这样的补丁是通过在图像上跨过而创建的,并且由于每个补丁都使用内核创建一个数字,因此您最终会得到一个包含图像中所有补丁的数字矩阵。

这是针对每张图像完成的,因此您最终会为每张图像得到一个较小的卷积图像。

代码示例

images = np.ones((2,3,4,4)) 
kernal = np.ones((3,3,3)) 
w = 4 
f = 3 
p = 0 
s = 1 
r = np.ones((2,  
        int((w-f+2*p)/s +1), int((w-f+2*p)/s +1))) 
for i, image in enumerate(images): 
    for h in range((4//3)+1): 
        for w in range((4//3)+1):             
            x = np.sum(image[:, w:w+3,h:h+3]*kernal) 
            r[i,w,h] = x 
print (r) 

输出:

[[[27. 27.] 
  [27. 27.]] 
 
 [[27. 27.] 
  [27. 27.]]] 

2 张大小为 4x4 的图像与大小为 3x3 的内核卷积会给你 2 张大小为 2x2 的图像(验证 (4 -3+0/1 +1, 4-3+0/1 +1))

必读资源:CV231n