Skip to main content
 首页 » 编程设计

python之使用来自谷歌驱动器的数据在 colab 中使用 fastai 创建数据束

2024年11月24日24jackei

我正在尝试使用 fast.ai 从我的 google drive 帐户加载 google colab 中的数据集。 我使用来自 kaggle 的外星人与捕食者作为数据集,来自 here 我下载并加载到我的谷歌驱动器中。然后我运行这段代码:

# Load the Drive helper and mount 
from google.colab import drive 
drive.mount("/content/drive") 
 
%reload_ext autoreload 
%autoreload 2 
%matplotlib inline 
from fastai import * 
from fastai.vision import * 
 
path='/content/drive/My Drive/FastaiData/Alien-vs-Predator' 
 
tfms = get_transforms(do_flip=False) 
 
#default bs=64, set image size=100 to run successfully on colab 
data = ImageDataBunch.from_folder(path,ds_tfms=tfms, size=100) 
path='/content/drive/My Drive/FastaiData/Alien-vs-Predator' 
 
tfms = get_transforms(do_flip=False) 
 
#default bs=64, set image size=100 to run successfully on colab 
data = ImageDataBunch.from_folder(path,ds_tfms=tfms, size=100) 

然后我得到了这个错误:

/usr/local/lib/python3.6/dist-packages/fastai/data_block.py:399: UserWarning: Your training set is empty. Is this is by design, pass `ignore_empty=True` to remove this warning. 
  warn("Your training set is empty. Is this is by design, pass `ignore_empty=True` to remove this warning.") 
/usr/local/lib/python3.6/dist-packages/fastai/data_block.py:402: UserWarning: Your validation set is empty. Is this is by design, use `no_split()`  
                 or pass `ignore_empty=True` when labelling to remove this warning. 
  or pass `ignore_empty=True` when labelling to remove this warning.""") 
IndexError                                Traceback (most recent call last) 
 
<ipython-input-3-5b3f66a4d360> in <module>() 
      4  
      5 #default bs=64, set image size=100 to run successfully on colab 
----> 6 data = ImageDataBunch.from_folder(path,ds_tfms=tfms, size=100) 
      7  
      8 data.show_batch(rows=3, figsize=(10,10)) 
 
/usr/local/lib/python3.6/dist-packages/fastai/vision/data.py in from_folder(cls, path, train, valid, valid_pct, classes, **kwargs) 
    118         if valid_pct is None: src = il.split_by_folder(train=train, valid=valid) 
    119         else: src = il.random_split_by_pct(valid_pct) 
--> 120         src = src.label_from_folder(classes=classes) 
    121         return cls.create_from_ll(src, **kwargs) 
    122  
 
 
and so on... 

它似乎发现我指定的文件夹为 train 和 validation set 是空的,这是不正确的。

谢谢你的帮助

请您参考如下方法:

将文件上传到 google drive 后,您应该使用 PyDrive。

示例片段。

!pip install -U -q PyDrive 
 
from pydrive.auth import GoogleAuth 
from pydrive.drive import GoogleDrive 
from google.colab import auth 
from oauth2client.client import GoogleCredentials 
 
# 1. Authenticate and create the PyDrive client. 
auth.authenticate_user() 
gauth = GoogleAuth() 
gauth.credentials = GoogleCredentials.get_application_default() 
drive = GoogleDrive(gauth) 
 
# PyDrive reference: 
# https://gsuitedevs.github.io/PyDrive/docs/build/html/index.html 
 
# 2. Create & upload a file text file. 
uploaded = drive.CreateFile({'title': 'Sample upload.txt'}) 
uploaded.SetContentString('Sample upload file content') 
uploaded.Upload() 
print('Uploaded file with ID {}'.format(uploaded.get('id'))) 
 
# 3. Load a file by ID and print its contents. 
downloaded = drive.CreateFile({'id': uploaded.get('id')}) 
print('Downloaded content "{}"'.format(downloaded.GetContentString())) 

详细引用请引用https://colab.research.google.com/notebooks/io.ipynb#scrollTo=zU5b6dlRwUQk