Skip to main content
 首页 » 编程设计

python之绕过 "Array is too big"python 错误

2024年11月01日7Terrylee

我有一个包含超过 200 000 行气象数据的 csv 文件。当我想用matplotlib对数据进行建模时,出现了这个错误:

Traceback (most recent call last): 
  File "try4.py", line 19, in <module> 
    X,Y = meshgrid( data_x,data_y )    
  File "C:\Python27\lib\site-packages\numpy\lib\function_base.py", line 3378, in  meshgrid 
    mult_fact = np.ones(shape, dtype=int)    
  File "C:\Python27\lib\site-packages\numpy\core\numeric.py", line 148, in ones 
    a = empty(shape, dtype, order)  
  ValueError: array is too big. 

我发现最多可以处理 5000 行的文件。 如何绕过错误以处理所有 200000 行的文件? 这是我的代码:

import numpy as np 
import matplotlib.pyplot as plt 
import matplotlib.mlab as mlab 
from pylab import * 
 
 
# read CSV as a numpy array 
data = mlab.csv2rec('datasets/mix.csv') 
 
# print CSV file headers 
print data.dtype.names 
 
# load columns as vectors 
data_x = data['longitude'] 
data_y = data['latitude'] 
data_u = data['x'] 
data_v = data['y'] 
 
X,Y = meshgrid( data_x,data_y ) 
U = cos(data_u) 
V = sin(data_v) 
 
 
# plot raw data 
Q = quiver( X, Y, U, V, units='width') 
qk = quiverkey(Q, 0.5, 0.92, 2, '.', labelpos='W', 
               fontproperties={'weight': 'bold'}) 
title('Current Surface') 
 
plt.show() 

请您参考如下方法:

为什么您使用 meshgrid ( doc )?它会很好地生成一个 200k x 200k 的数组,该数组将与您的 u 和 v 数据的尺寸不匹配。我想你想这样做

import numpy as np 
import matplotlib.pyplot as plt 
import matplotlib.mlab as mlab 
from pylab import * 
 
 
# read CSV as a numpy array 
data = mlab.csv2rec('datasets/mix.csv') 
 
# print CSV file headers 
print data.dtype.names 
 
# load columns as vectors 
data_x = data['longitude'] 
data_y = data['latitude'] 
data_u = data['x'] 
data_v = data['y'] 
 
U = cos(data_u) 
V = sin(data_v) 
 
 
# plot raw data 
Q = quiver(data_x, data_y, U, V, units='width') 
qk = quiverkey(Q, 0.5, 0.92, 2, '.', labelpos='W', 
               fontproperties={'weight': 'bold'}) 
title('Current Surface')