If you work with Tensorflow and want to write a custom layer as a subclass of tf.keras.layers.Layer, then there are some details which took me a while to figure out.
1) There are usually instructions to create the layer for Sequential API (tf.keras.Sequential) , and one of mandatory input parameters is input_shape. If you write your layer for Functional API (tf.keras.Model), then the corresponding parameter is shape. Thanks to the book "TensorFlow in Action" by Thushan Ganegedara for this!
2) The subclass may not calculate output tensor shape correctly when you do some transformations. If the tensor shape and/or its dimension count changes during the pass through your custom layer, add a method for output shape computation at the end, so your layer will call it and a next layer will work smoothly with it. The above book mentions compute_output_shape() method, although without detail. You can see an example of it below at the end. I found the correct format on stackoverflow:
https://stackoverflow.com/questions/60598421/tf-keras-custom-layer-output-shape-is-none .
About the code below: I marked some custom rows by ellipsis (no, I do not use Python Ellipses notation here). Variables param00 and param01 are your optional layer parameters. You may add build method to define initial variables or constants for the
layer which are needed at the start of the layer calculations as well, but it is
optional. The shape (or input_shape) parameter must be present if you use the layer at the start of a model or if your change a tensor shape during your layer pass.
class CustomLayer(tf.keras.layers.Layer):
self.shape = shape
...
def call(self, input_tensor):
if training:
...
...
return output_tensor
else:
return input_tensor
def compute_output_shape(self, shape):
return (input_shape[0], self.output_dim)
3) The training parameter has a boolean value and indicates if the layer is required for prediction or only for training. For example, you can apply custom transformations like uniform noise instead of Gaussian. The parameter is computed automatically and should not be used for anything else.
4) There are several other variables, parameters and methods which must be named just so and used in particular way. They are called inherited from the class. You can see some of them them in the Tensorflow documentation for the layer:
https://www.tensorflow.org/guide/keras/making_new_layers_and_models_via_subclassing
Of course self and training are on the list, but in addition the list contains parameters shape for functional API and input_shape for Sequential API, input_tensor, output_tensor and output_dim. If you keep getting weird error messages about your indentificators, this could be a reason. Here are couple of ways to deal with it:
a) You can look up Tensorflow code for the layer class.
b) If you intend to use the code only for yourself then a lazy way to fix the problem is switching your identificators to something not so PEP 8 and scikit-learn standards.
