python之如何在不保存在磁盘上的情况下将 pandas 数据帧上传为 csv 流

我想将 pandas 数据帧作为 csv 文件上传到服务器，而不将其保存在磁盘上。有没有办法创建一个或多或少的“假 csv”文件来伪装成真实文件？

下面是一些示例代码:

首先，我从 sql 查询中获取数据并将其存储为数据框。 在 upload_ga_data 函数中，我想使用以下逻辑:

 media = MediaFileUpload('df', 
                      mimetype='application/octet-stream', 
                      resumable=False) 

完整示例:

from __future__ import print_function 
from apiclient.discovery import build 
from oauth2client.service_account import ServiceAccountCredentials 
from googleapiclient.errors import HttpError 
from apiclient.http import MediaFileUpload 
import pymysql 
import pandas as pd 
con = x 
 
ga_query = """ 
    SELECT XXXXX 
    """ 
 
df = pd.read_sql_query(ga_query,con) 
 
df.to_csv('ga_export.csv', sep=',', encoding='utf-8', index = False) 
 
def upload_ga_data(): 
    try: 
        media = MediaFileUpload('ga_export.csv', 
                          mimetype='application/octet-stream', 
                          resumable=False) 
        daily_upload = service.management().uploads().uploadData( 
                accountId=accountId, 
                webPropertyId=webPropertyId, 
                customDataSourceId=customDataSourceId, 
                media_body=media).execute() 
        print ("Upload was successfull") 
    except TypeError as error: 
      # Handle errors in constructing a query. 
      print ('There was an error in constructing your query : %s' % error) 

请您参考如下方法:

所需的行为可以使用 stream 实现:

to create a more or less "fake csv" file which pretends to be a real file

Python 使文件描述符(使用open)和Stream(使用io.StringIO)的行为相似。然后在任何可以使用文件描述符的地方也可以使用字符串流。

The easiest way to create a text stream is with open(), optionally specifying an encoding:

f = open("myfile.txt", "r", encoding="utf-8") 

In-memory text streams are also available as StringIO objects:

f = io.StringIO("some initial text data") 

The text stream API is described in detail in the documentation of TextIOBase.

在 Pandas 中，您可以使用任何 function在其 signature 中包含 path_or_buf 参数，例如to_csv:

DataFrame.to_csv(path_or_buf=None, sep=', ', na_rep='', float_format=None, columns=None, header=True, index=True, index_label=None, mode='w', encoding=None, compression=None, quoting=None, quotechar='"', line_terminator='\n', chunksize=None, tupleize_cols=None, date_format=None, doublequote=True, escapechar=None, decimal='.')

以下代码将 CSV 格式的虚拟 DataFrame 导出到字符串流(不是物理文件，内存中的八位字节流):

import io 
import pandas as pd 
 
df = pd.DataFrame(list(range(10))) 
 
stream = io.StringIO() 
df.to_csv(stream, sep=";") 

当您想访问流内容时，只需发出:

>>> stream.getvalue() 
';0\n0;0\n1;1\n2;2\n3;3\n4;4\n5;5\n6;6\n7;7\n8;8\n9;9\n' 

它无需使用真实文件即可返回内容。