Skip to main content
 首页 » 编程设计

python之将 csv 写入谷歌云存储

2025年02月15日9zhengyun_ustc

我想了解如何将多行 csv 文件写入谷歌云存储。我只是没有关注 documentation

靠近这里: Unable to read csv file uploaded on google cloud storage bucket

例子:

from google.cloud import storage 
from oauth2client.client import GoogleCredentials 
import os 
 
os.environ['GOOGLE_APPLICATION_CREDENTIALS'] = "<pathtomycredentials>" 
 
a=[1,2,3] 
 
b=['a','b','c'] 
 
storage_client = storage.Client() 
bucket = storage_client.get_bucket("<mybucketname>") 
 
blob=bucket.blob("Hummingbirds/trainingdata.csv") 
 
for eachrow in range(3): 
    blob.upload_from_string(str(a[eachrow]) + "," + str(b[eachrow])) 

这让你在谷歌云存储上单线

3,c 

很明显,它每次都打开一个新文件并写入该行。

好的,添加一个新行 delim 怎么样?

for eachrow in range(3): 
    blob.upload_from_string(str(a[eachrow]) + "," + str(b[eachrow]) + "\n") 

添加了换行符,但还是从头开始写。

有人可以说明这种方法是什么吗?我可以将我所有的行组合成一个字符串,或者写一个临时文件,但这看起来很难看。

也许以文件形式打开?

请您参考如下方法:

请引用以下答案,希望对您有所帮助。

import pandas as pd 
 data = [['Alex','Feb',10],['Bob','jan',12]] 
 df = pd.DataFrame(data,columns=['Name','Month','Age']) 
 print df 

输出

   Name Month  Age 
0  Alex   Feb   10 
1   Bob   jan   12 

添加一行

row = ['Sally','Oct',15] 
df.loc[len(df)] = row 
print df 

输出

     Name Month  Age 
 0   Alex   Feb   10 
 1    Bob   jan   12 
 2  Sally   Oct   15 

使用 gsutil 写入/复制到 GCP Bucket

  df.to_csv('text.csv', index = False) 
 !gsutil cp 'text.csv' 'gs://BucketName/folderName/' 

Python 代码(文档 https://googleapis.dev/python/storage/latest/index.html)

from google.cloud import storage 
 
def upload_to_bucket(bucket_name, blob_path, local_path): 
    bucket = storage.Client().bucket(bucket_name) 
    blob = bucket.blob(blob_path) 
    blob.upload_from_filename(local_path) 
    return blob.url 
 
# method call 
bucket_name = 'bucket-name' # do not give gs:// ,just bucket name 
blob_path = 'path/folder name inside bucket' 
local_path = 'local_machine_path_where_file_resides' #local file path 
upload_to_bucket(bucket_name, blob_path, local_path)