Skip to main content
 首页 » 编程设计

python之无法保留内存块,在 python 中导入 json 错误

2024年12月31日8zengkefu

import pandas as pd 
with open(r'data.json') as f: 
   df = pd.read_json(f, encoding='utf-8') 

我收到“无法保留内存块”错误。 JSON 文件的大小为 300MB。在 Python 中为正在运行的程序保留内存是否有任何限制?我在 PC 上有 8GB RAM,使用 Windows 10。

loading of json file into df 
Traceback (most recent call last): 
  File "C:\Program Files\JetBrains\PyCharm 2018.1.4\helpers\pydev\pydev_run_in_console.py", line 52, in run_file 
    pydev_imports.execfile(file, globals, locals)  # execute the script 
  File "C:\Program Files\JetBrains\PyCharm 2018.1.4\helpers\pydev\_pydev_imps\_pydev_execfile.py", line 18, in execfile 
    exec(compile(contents+"\n", file, 'exec'), glob, loc) 
  File "C:/Users/Beorn/PycharmProjects/project_0/projekt/test.py", line 7, in <module> 
    df = pd.read_json(f, encoding='utf-8') 
  File "C:\Users\Beorn\AppData\Local\Programs\Python\Python36-32\lib\site-packages\pandas\io\json\json.py", line 422, in read_json 
    result = json_reader.read() 
  File "C:\Users\Beorn\AppData\Local\Programs\Python\Python36-32\lib\site-packages\pandas\io\json\json.py", line 529, in read 
    obj = self._get_object_parser(self.data) 
  File "C:\Users\Beorn\AppData\Local\Programs\Python\Python36-32\lib\site-packages\pandas\io\json\json.py", line 546, in _get_object_parser 
    obj = FrameParser(json, **kwargs).parse() 
  File "C:\Users\Beorn\AppData\Local\Programs\Python\Python36-32\lib\site-packages\pandas\io\json\json.py", line 638, in parse 
    self._parse_no_numpy() 
  File "C:\Users\Beorn\AppData\Local\Programs\Python\Python36-32\lib\site-packages\pandas\io\json\json.py", line 853, in _parse_no_numpy 
    loads(json, precise_float=self.precise_float), dtype=None) 
ValueError: Could not reserve memory block 
PyDev console: starting. 
Python 3.6.6 (v3.6.6:4cf1f54eb7, Jun 27 2018, 02:47:15) [MSC v.1900 32 bit (Intel)] on win32 

请您参考如下方法:

因此,在阅读了大量帖子和解决方案之后,我决定通过删除无用数据来减小文件大小。也许你会发现这很有用。顺便说一下,我在某处读到你需要的内存至少比你的 JSON 文件多 25 倍,所以在我的例子中,我需要超过 8GB。

with open('data.json', 'r') as data_file: 
    data = json.load(data_file) 
 
print(data.keys()) 
del data['author'] 
 
with open('datav2.json', 'w') as data_file: 
    data = json.dump(data, data_file)