我想从磁盘上的文件同时加载多个灰度图像,并将它们放在一个大的 numpy 数组中,以加快加载时间。基本代码如下所示:
import numpy as np
import matplotlib.pyplot as plt
# prepare filenames
image_files = ...
mask_files = ...
n_samples = len(image_files) # == len(mask_files)
# preallocate space
all_images = np.empty(shape=(n_samples, IMG_HEIGHT, IMG_WIDTH), dtype=np.float32)
all_masks = np.empty(shape=(n_samples, IMG_HEIGHT, IMG_WIDTH), dtype=np.float32)
# read images and masks
for sample, (img_path, mask_path) in enumerate(zip(image_files, mask_files)):
all_images[sample, :, :] = plt.imread(img_path)
all_masks[sample, :, :] = plt.imread(mask_path)
我想并行执行这个循环,但是,我知道 Python 真正的多线程能力由于 GIL 而受到限制。
你有什么想法吗?
请您参考如下方法:
你可以尝试对图像做一个,对蒙版做一个
import numpy as np
import matplotlib.pyplot as plt
from threading import Thread
# threading functions
def readImg(image_files, mask_files):
for sample, (img_path, mask_path) in enumerate(zip(image_files, mask_files)):
all_images[sample, :, :] = plt.imread(img_path)
def readMask(image_files, mask_files):
for sample, (img_path, mask_path) in enumerate(zip(image_files, mask_files)):
all_masks[sample, :, :] = plt.imread(mask_path)
# prepare filenames
image_files = ...
mask_files = ...
n_samples = len(image_files) # == len(mask_files)
# preallocate space
all_images = np.empty(shape=(n_samples, IMG_HEIGHT, IMG_WIDTH), dtype=np.float32)
all_masks = np.empty(shape=(n_samples, IMG_HEIGHT, IMG_WIDTH), dtype=np.float32)
# threading stuff
image_thread = Thread(target=readImg,
args=[image_files, mask_files])
mask_thread = Thread(target=readMask,
args=[image_files, mask_files])
image_thread.daemon = True
mask_thread.daemon = True
image_thread.start()
mask_thread.start()
警告:请勿复制此代码。我也没有对此进行测试,只是为了了解它的要点。
这不会使用多核,也不会像上面的代码那样线性执行。如果你想要,你必须使用队列实现。虽然,我认为这不是您想要的,因为您说您想要并发并且知道 python 线程上的解释器锁。
编辑 - 根据您的评论,请参阅这篇关于使用多核的帖子 Multiprocessing vs Threading Python , 要对上述示例进行更改,只需使用行
import multiprocessing.Process as Thread
它们共享相似的 API。
