我有一个图像链接列表(大约 5000 行),我需要知道如何快速下载所有图像。请帮助我的代码:
import concurrent.futures
import urllib.request
catname = 'amateur'
def getimg (count, endcount):
while (count < endcount):
urllib.request.urlretrieve(URLS[count], catname+'/images/'+catname+str(count)+'.jpg')
URLS[count] = catname+'/images/'+catname+str(count)+'.jpg'
count = count + 1
with concurrent.futures.ThreadPoolExecutor(max_workers=50) as e:
e.submit(getimg, 0, 5000)
它工作正常但速度很慢。
请您参考如下方法:
您的代码下载 5000 张图像 50 次。尝试以下操作:
import concurrent.futures
import urllib.request
catname = 'amateur'
def getimg(count):
localpath = '{0}/images/{0}{1}.jpg'.format(catname, count)
urllib.request.urlretrieve(URLS[count], localpath)
URLS[count] = localpath
with concurrent.futures.ThreadPoolExecutor(max_workers=50) as e:
for i in range(5000):
e.submit(getimg, i)
