Skip to main content
 首页 » 编程设计

python之带有 Bokeh nyc_taxi 示例的 MemoryError

2025年01月19日9txw1958

我试着按照这里的例子:https://anaconda.org/jbednar/nyc_taxi/notebook

但是,我无法让以下 block 工作,因为 MemoryError 总是在特定行抛出(已注释掉):

def merged_images(x_range, y_range, w=plot_width, h=plot_height, how='log'): 
    cvs = ds.Canvas(plot_width=w, plot_height=h, x_range=x_range, y_range=y_range) 
    picks = cvs.points(df, 'pickup_x',  'pickup_y',  ds.count('passenger_count')) 
    drops = cvs.points(df, 'dropoff_x', 'dropoff_y', ds.count('passenger_count')) 
    #more_drops = tf.shade(drops.where(drops > picks), cmap=["darkblue", 'cornflowerblue'], how=how) 
    #more_picks = tf.shade(picks.where(picks > drops), cmap=["darkred", 'orangered'],  how=how) 
    img = tf.stack(more_picks,more_drops) 
    return tf.dynspread(img, threshold=0.3, max_px=4) 
 
p = base_plot(background_fill_color=background) 
export(merged_images(*NYC),"NYCT_pickups_vs_dropoffs") 
InteractiveImage(p, merged_images) 

这里需要大量 RAM (>64GB),还是我遗漏了一些与内存相关的配置?我已经尝试在 Windows 10 和 Linux 16.04(均为 64 位版本)上使用当前版本的 Python 3.6 和相应的库(bokeh、datashader、jupyter)无济于事。

更新:我还注意到,即使我的 df.tail() 似乎符合(11842093 条记录),直方图结果(从 histogram(agg.values) 之后)与原始笔记本(截至 https://anaconda.org/jbednar/nyc_taxi/notebook?version=2016.08.18.1303)有很大不同。

请您参考如下方法:

根据@JamesA.Bednar 的评论:和相关提交:https://github.com/bokeh/datashader/commit/9fbace5c7b00410bdac7b7662ee24e466bc66330 , 问题出现在 xarray>=0.8

修复是“重命名列以在比较/合并/连接之前匹配”

结果:

def merged_images(x_range, y_range, w=plot_width, h=plot_height, how='log'): 
    cvs = ds.Canvas(plot_width=w, plot_height=h, x_range=x_range, y_range=y_range) 
    picks = cvs.points(df, 'pickup_x',  'pickup_y',  ds.count('passenger_count')) 
    drops = cvs.points(df, 'dropoff_x', 'dropoff_y', ds.count('passenger_count')) 
    drops = drops.rename({'dropoff_x': 'x', 'dropoff_y': 'y'}) # added line 
    picks = picks.rename({'pickup_x': 'x', 'pickup_y': 'y'}) # added line 
    more_drops = tf.shade(drops.where(drops > picks), cmap=["darkblue", 'cornflowerblue'], how=how) 
    more_picks = tf.shade(picks.where(picks > drops), cmap=["darkred", 'orangered'],  how=how) 
    img = tf.stack(more_picks,more_drops) 
    return tf.dynspread(img, threshold=0.3, max_px=4)