Fix error: "Object of type int32 is not JSON serializable" when saving pipeline
Created by: rozlana-g
Прогоняла automl класс на моих TS рядах (всего 12 рядов) в цикле таким образом:
for ts_id_ in ts_ids:
...
model = Fedot(problem='ts_forecasting', task_params=task_parameters, timeout=25, verbose_level=4)
pipeline = model.fit(features=train_input)
forecast = model.predict(features=predict_input)
pipeline.save(path=f'./fedot_forecasts/cycle_92_25min2/{ts_id_}/')Прогоняла этот цикл по моим TS рядам несколько (~5-6) раз. Иногда сталкивалась с такой ошибкой при сохранении пайплайна :
The pipeline saved in the path: C:\Users\user\Fedot\composing_history_1633802462.758717\5\2b9c9a85-baea-47e5-a01d-d65732b8f605\2b9c9a85-baea-47e5-a01d-d65732b8f605.json.
spent time: 14.4 min
Best metric is 11.597088925380861
Result:
Best metric is 11.597088925380861
Composition time: 14.374 min
Algorithm was terminated due to processing time limit
GP composition finished
Hyperparameters tuning started
Start pipeline tuning
Hyperparameters optimization start
Fit pipeline from scratch
0%| | 0/1000 [00:00<?, ?trial/s, best loss=?]Fit pipeline from scratch
0%|1 | 1/1000 [04:45<79:08:10, 285.18s/trial, best loss: 12.466931623329755]Fit pipeline from scratch
0%|2 | 2/1000 [05:51<43:23:03, 156.50s/trial, best loss: 12.466931623329755]Fit pipeline from scratch
0%|3 | 3/1000 [1:12:00<525:25:18, 1897.21s/trial, best loss: 11.384835428959343]
0%|3 | 3/1000 [1:12:00<398:49:25, 1440.09s/trial, best loss: 11.384835428959343]
Fit pipeline from scratch
Hyperparameters optimization finished
Return tuned pipeline due to the fact that obtained metric 11.385 equal or smaller than initial (+ 5% deviation) 12.177
Tuning was finished
Model composition finished
Fit pipeline from scratch
Object of type int32 is not JSON serializable
2b9c9a85-baea-47e5-a01d-d65732b8f605.txt
The pipeline saved in the path: C:\Users\user\Fedot\composing_history_1633818918.9425\2\d42ba2a0-478d-4bce-aa8f-b26a16fbc88b\d42ba2a0-478d-4bce-aa8f-b26a16fbc88b.json.
spent time: 11.5 min
Best metric is 28.045402352010672
Result:
Best metric is 28.045402352010672
Composition time: 11.48 min
Algorithm was terminated due to processing time limit
GP composition finished
Hyperparameters tuning started
Start pipeline tuning
Hyperparameters optimization start
Fit pipeline from scratch
0%| | 0/1000 [00:00<?, ?trial/s, best loss=?]Fit pipeline from scratch
0%|1 | 1/1000 [00:39<10:53:28, 39.25s/trial, best loss: 26.560896302684963]Fit pipeline from scratch
0%|2 | 2/1000 [06:52<65:16:39, 235.47s/trial, best loss: 26.560896302684963]Fit pipeline from scratch
0%|3 | 3/1000 [07:30<40:18:36, 145.55s/trial, best loss: 26.560896302684963]Fit pipeline from scratch
0%|4 | 4/1000 [18:07<93:58:46, 339.69s/trial, best loss: 26.560896302684963]
0%|4 | 4/1000 [18:07<75:14:51, 271.98s/trial, best loss: 26.560896302684963]
Fit pipeline from scratch
Hyperparameters optimization finished
Return tuned pipeline due to the fact that obtained metric 26.727 equal or smaller than initial (+ 5% deviation) 29.482
Tuning was finished
Model composition finished
Fit pipeline from scratch
Object of type int32 is not JSON serializabled42ba2a0-478d-4bce-aa8f-b26a16fbc88b.txt
Поняла, что ошибка вылезает во время выполнения json.dumps, когда в атрибуты пайпа попадают данные с типом np.integer (np.int32). Сделала encoder для numpy типов:
class NumpyIntEncoder(json.JSONEncoder):
def default(self, obj):
if isinstance(obj, np.integer):
return int(obj)
return json.JSONEncoder.default(self, obj)