我正在尝试在 for 循环中针对不同的响应变量运行多个 H2O 模型。
H2O cluster uptime: 53 mins 11 secs
H2O cluster timezone: Etc/UTC
H2O data parsing timezone: UTC
H2O cluster version: 3.22.1.1
H2O cluster version age: 2 hours and 15 minutes
H2O cluster name: H2O_from_python_root_np3l2m
H2O cluster total nodes: 1
H2O cluster free memory: 13.01 Gb
H2O cluster total cores: 8
H2O cluster allowed cores: 8
H2O cluster status: locked, healthy
H2O connection url: http://localhost:54321
H2O connection proxy:
H2O internal security: False
H2O API Extensions: XGBoost, Algos, AutoML, Core V3, Core V4
Python version: 2.7.12 final
我已经为选择训练集/验证集和模型本身设置了一个种子。我有提前停止事件,但根据文档,只要 score_tree_interval 处于事件状态,结果应该是可重现的。
### This is the code that's defining the model
def append_probs(hframe, response_col, model):
pd_df = h2o.as_list(hframe).copy()
pd_df.loc[:,'pred'] = h2o.as_list(model.predict(hframe)).values
pd_df.loc[:,'error'] = pd_df['pred'] - pd_df[response_col]
return pd_df
def run_model(response_col, model_typ, hframe_train, hframe_pred):
h2o_dtypes = [hframe_train.type(e) for e in hframe_train.columns]
data = h2o.deep_copy(hframe_train,'data')
mapping = {'new_email_ldsub':'live_pp',
'new_call_ldsub':'live_pp',
'used_email_ldsub':'live_usedplus',
'used_call_ldsub':'live_usedplus',
'myapp_edm_ldsub':'live_myapp',
'cc_edm_ldsub':'live_cc',
'fbm_call_ldsub':'live_fbm',
'fbm_email_ldsub':'live_fbm'}
data = data[data[mapping[response_col]]==1]
train, valid = data.split_frame([0.8], seed=1234)
X = hframe_train.col_names[:-14]
print X
y = response_col
print y
if model_typ == 'gbm':
model = H2OGradientBoostingEstimator(
ntrees=512,
learn_rate=0.08,
max_depth=7,
col_sample_rate = 0.7,
sample_rate = 0.9,
stopping_tolerance=1e-05,
stopping_rounds=2,
score_tree_interval=5,
#nfolds=5,
#fold_assignment = "Random",
distribution = 'poisson',
seed=20000,
stopping_metric='mae',
min_rows = 10,
nbins = 30
model.train(X, y, training_frame=train, validation_frame=valid)
pred_df = append_probs(hframe_pred,response_col,model)
return model, pred_df
### This is the code that runs the model
gbm_results = pd.DataFrame()
gbm_mapping = {'live_pp':['new_call_ldsub','new_email_ldsub'],
'live_usedplus':['used_call_ldsub','used_email_ldsub'],
'live_myapp':['myapp_edm_ldsub'],
'live_cc':['cc_edm_ldsub'],
'live_fbm':['fbm_call_ldsub','fbm_email_ldsub']}
gbm_train_err = {}
gbm_valid_err = {}
gbm_xval_err = {}
for k,v in gbm_mapping.iteritems():
for e in v:
gbm_mod, gbm_pred_df = run_model(e,'gbm',hframe,hframe_forecast_pred)
gbm_pred_df = gbm_pred_df[['id','month','pred']]
gbm_pred_df = gbm_pred_df.groupby(['id','month'])['pred'].sum().reset_index()
gbm_pred_df.loc[:,'product'] = str(e)
gbm_train_err[str(e)] = [gbm_mod.mae(train=True),gbm_mod.rmse(train=True)]
gbm_valid_err[str(e)] = [gbm_mod.mae(valid=True),gbm_mod.rmse(valid=True)]
gbm_xval_err[str(e)] = [gbm_mod.mae(xval=True),gbm_mod.rmse(xval=True)]
gbm_results = pd.concat([gbm_results, gbm_pred_df])
gbm_results['process_month'] = pd.to_datetime(gbm_results['process_month'],unit='ms')
根据文档,我希望每个模型的结果都是可重现的/接近的。
请您参考如下方法:
自最新版本的 H2O-3 3.22.1.1 起,重现性要求列在文档中 here .
为方便起见,以下是模型在单个节点上的可重复性要求:
请注意,除了种子之外,您还需要使用相同的数据(相同的拆分)、相同的参数,并且要么不使用早停,要么使用设置了 score_tree_interval 的早停。
如何保证单节点集群的可重现性?
必须满足以下条件才能保证单节点集群中的可再现性:
- 相同的训练数据
注意:如果您使用 H2O 导入包含多个文件而不是单个文件的整个目录,我们不保证再现性,因为数据可能在导入过程中被打乱。
- 用于训练模型的相同参数
- 相同的种子集(完成任何采样时都需要)
- 没有提前停止或使用 score_tree_interval 设置和相同的验证数据提前停止
