Python error while doing tf.keras hyperparameter optimization

I’m using keras-tuner in order to do an hyperparameter optimization of a neural network.

I’m using an Hyperband optimization, implemented by this Python function:

import keras_tuner as kt def tuning_function(self): objective = Objective('val_loss', direction="min") tuner = kt.Hyperband(ann_model, objective=objective, max_epochs=100, factor=2, directory=/path/to/folder, project_name="name", seed=0) stop_early = tf.keras.callbacks.EarlyStopping(monitor='val_loss', min_delta=0.1, mode='min', patience=15) tensorboard = TensorBoard(log_dir=log_dir) tuner.search(training_gen(), epochs=50, validation_data=valid_gen(), callbacks=[stop_early, tensorboard], steps_per_epoch=np.round(int(total_num_samples / batch_size), decimals=0), validation_freq=1, validation_steps=100) 

where ann_model is the compiled model of the Artificial Neural Network under test, while both training_gen() and valid_gen() are Python generators.

As it can be seen, Early Stopping and Tensorbord are the callbacks passed to tuner.search().

The Keras Hyperband algorithm works well: it simulates the various models, and when it reaches the Early Stopping condition on a model, the training of that model stops and the training of a new model starts; the point is that I noticed that between these two events (stop and start), the following two consecutive Python errors occur (anyway, the simulation doesn’t exit or generate an Exception, but it goes on, by simulating the successive model):

W tensorflow/core/framework/op_kernel.cc:1755] Invalid argument: ValueError: callback pyfunc_2 is not found Traceback (most recent call last): File "/home/username/anaconda3/envs/myenv/lib/python3.8/site-packages/tensorflow/python/ops/script_ops.py", line 233, in __call__ raise ValueError("callback %s is not found" % token) ValueError: callback pyfunc_2 is not found W tensorflow/core/kernels/data/generator_dataset_op.cc:103] Error occurred when finalizing GeneratorDataset iterator: Invalid argument: ValueError: callback pyfunc_2 is not found Traceback (most recent call last): File "/home/username/anaconda3/envs/myenv/lib/python3.8/site-packages/tensorflow/python/ops/script_ops.py", line 233, in __call__ raise ValueError("callback %s is not found" % token) ValueError: callback pyfunc_2 is not found [[{{node PyFunc}}]] 

The first error seems to refer to a “callback”, while the second one refers to a “callback” and a “generator”.

Which could be the cause of these errors? What could be this callback “not found”?