nevopy.fixed_topology.layers package¶

Submodules¶

nevopy.fixed_topology.layers.base_layer module¶

Defines the abstract class that serves as a base for all the neural layers used by fixed-topology neuroevolutionary algorithms.

class nevopy.fixed_topology.layers.base_layer.BaseLayer(config=None, input_shape=None, mutable=True)¶

Bases: abc.ABC

Abstract base class that defines a neural network layer.

This abstract base class defines the general structure and behaviour of a fixed topology neural network layer in the context of neuroevolutionary algorithms.

Parameters

config (Optional[FixedTopologyConfig]) – Settings being used in the current evolutionary session. If None, a config object must be assigned to the layer later on, before calling the methods that require it.
input_shape (Optional[Tuple[int, ..]]) – Shape of the data that will be processed by the layer. If None, an input shape for the layer must be manually specified later or be inferred from an input sample.
mutable (Optional[bool]) – Whether or not the layer can have its weights changed (mutation).

config¶

Settings being used in the current evolutionary session. If None, a config object hasn’t been assigned to the layer yet.

Type: Optional[FixedTopologyConfig]

mutable¶

Whether or not the layer can have its weights changed (mutation).

Type: bool

abstract build(input_shape)¶

Builds the layer’s weight and bias matrices.

If the layer has already been built, it will be built again (new weight and bias matrices will be generated).

Parameters: input_shape (Tuple[int, ..]) – Tuple with the shape of the inputs that will be fed to the layer.
Raises: ValueError – If the layer isn’t compatible with the given input shape.
Return type: None

abstract deep_copy()¶

Makes an exact/deep copy of the layer.

Return type: BaseLayer
Returns: An exact/deep copy of the layer, including its weights and biases.

property input_shape¶

The expected shape of an input for the layer.

Return type: Optional[Tuple[int, …]]
Returns: A tuple with the layer’s input shape or None if an input shape hasn’t been specified yet.

classmethod load(abs_path)¶

Loads the layer from the given absolute path.

This method uses, by default, pickle to load the layer.

Parameters: abs_path (str) – Absolute path of the saved “.pkl” file. If the given path doesn’t end with the suffix “.pkl”, it will be automatically added to it.
Return type: BaseLayer
Returns: The loaded layer.

abstract mate(other)¶

Mates two layers to produce a new layer (offspring).

Implements the sexual reproduction between a pair of layers. The new layer inherits information from both parents (not necessarily in an equal proportion)

Parameters: other (Any) – The second layer . If it’s not compatible for mating with the current layer (self), an exception will be raised.
Return type: BaseLayer
Returns: A new layer (the offspring born from the sexual reproduction between the current layer and the layer passed as argument. If the layer is immutable, other is expected to be equal to self, so a deep copy (BaseLayer.deep_copy()) of the layer is returned.
Raises: IncompatibleLayersError – If the layer passed as argument to other is incompatible with the current layer (self).

abstract mutate_weights()¶

Randomly mutates the weights of the layer’s connections.

If the layer is immutable, this method doesn’t do anything.

Return type: None

abstract process(x)¶

Feeds the given input(s) to the layer.

This is where the layer’s logic lives. If the layer hasn’t been built yet, it will be automatically built using the given input shape.

Parameters: x (Any) – The input(s) to be fed to the layer. Usually a NumPy ndarray or a TensorFlow tensor.
Return type: Any
Returns: The output of the layer. Usually a NumPy ndarray or a TensorFlow tensor.
Raises: InvalidInputError – If the shape of x doesn’t match the input shape expected by the layer.

abstract random_copy()¶

Makes a random copy of the layer.

Return type: BaseLayer
Returns: A new layer with the same topology of the current layer, but with newly initialized weights and biases. If the layer is immutable, a deep copy (BaseLayer.deep_copy()) of the layer is returned instead.

save(abs_path)¶

Saves the layer on the absolute path provided.

This method uses, by default, pickle to save the layer.

Parameters: abs_path (str) – Absolute path of the saving file. If the given path doesn’t end with the suffix “.pkl”, it will be automatically added to it.
Return type: None

abstract property weights¶

A list with the layer’s weight matrices as numpy arrays.

For most layers, it’s a tuple containing a weight matrix and a bias vector.

Return type: List[ndarray]

exception nevopy.fixed_topology.layers.base_layer.IncompatibleLayersError¶

Bases: Exception

Indicates that an attempt has been made to mate (sexual reproduction) two incompatible layers.

nevopy.fixed_topology.layers.mating module¶

Implements some mating (sexual reproduction) functions that can be used to generate a new neural network layer from two parent layers.

nevopy.fixed_topology.layers.mating.check_weights_compatibility(weight_list1, weight_list2)¶

Checks the mating compatibility between two lists of weight matrices.

Raises: IncompatibleLayersError – If one or more weight matrices in one of the lists don’t have the same shape as the corresponding weight matrices in the other list.

nevopy.fixed_topology.layers.mating.exchange_units_mating(layer1, layer2)¶

Mates (sexual reproduction) two neural layers by exchanging units.

The term “unit” means different things depending on the type of the layers. For a Conv2D layer, for instance, an unit is a filter (kernel). For a Dense layer, on the other hand, an unit is a neuron (including the weights of its connections). Bias terms are also considered “units”.

Generally, we can define an unit as being whatever you get when indexing a weight matrix by its last shape. Given a layer L, an unit of its weight matrix at index w is given by L.weights[w][…, i], where i is the index of the unit (i is in the interval [0, w.shape[-1][).

Parameters

layer1 (BaseLayer) – An instance of a subclass of BaseLayer.
layer2 (BaseLayer) – An instance of a subclass of BaseLayer.

Return type

BaseLayer

Returns

A new layer that inherits information from both parents.

Raises

IncompatibleLayersError – If the weight matrices of the two given layers are not of the same shape (i.e., the layers are not compatible for mating).

nevopy.fixed_topology.layers.mating.exchange_weights_mating(layer1, layer2)¶

Mates (sexual reproduction) two neural layers by exchanging weights.

Each of the new layer’s weights is randomly inherited, with equal chance, from one of the parent layers.

Parameters

layer1 (BaseLayer) – An instance of a subclass of BaseLayer.
layer2 (BaseLayer) – An instance of a subclass of BaseLayer.

Return type

BaseLayer

Returns

A new layer that randomly inherits its individual weights from its parent layers.

Raises

IncompatibleLayersError – If the weight matrices of the two given layers are not of the same shape (i.e., the layers are not compatible for mating).

nevopy.fixed_topology.layers.mating.weights_avg_mating(layer1, layer2)¶

Mates (sexual reproduction) two layers by averaging their weights.

Each of the new layer’s weight is the simple average (sum and divide by 2) of the parent layers weights.

Parameters

layer1 (BaseLayer) – An instance of a subclass of BaseLayer.
layer2 (BaseLayer) – An instance of a subclass of BaseLayer.

Return type

BaseLayer

Returns

A new layer whose weights are the simple average of the parent layers weights.

Raises

IncompatibleLayersError – If the weight matrices of the two given layers are not of the same shape (i.e., the layers are not compatible for mating).

nevopy.fixed_topology.layers.tf_layers module¶

Implements subclasses of BaseLayer that wrap TensorFlow layers.

class nevopy.fixed_topology.layers.tf_layers.TFConv2DLayer(filters, kernel_size, strides=(1, 1), padding='valid', activation='relu', mating_func=<function exchange_units_mating>, config=None, input_shape=None, mutable=True, **tf_kwargs)¶

Bases: nevopy.fixed_topology.layers.tf_layers.TensorFlowLayer

Wraps a TensorFlow 2D convolution layer.

This is a simple wrapper for tf.keras.layers.Conv2D.

class nevopy.fixed_topology.layers.tf_layers.TFDenseLayer(units, activation=None, mating_func=<function exchange_weights_mating>, config=None, input_shape=None, mutable=True, **tf_kwargs)¶

Bases: nevopy.fixed_topology.layers.tf_layers.TensorFlowLayer

Wraps a TensorFlow dense layer.

This is a simple wrapper for tf.keras.layers.Dense.

class nevopy.fixed_topology.layers.tf_layers.TFFlattenLayer(mating_func=None, config=None, input_shape=None, mutable=False, **tf_kwargs)¶

Bases: nevopy.fixed_topology.layers.tf_layers.TensorFlowLayer

Wraps a TensorFlow flatten layer.

This is a simple wrapper for tf.keras.layers.Flatten.

class nevopy.fixed_topology.layers.tf_layers.TFMaxPool2DLayer(pool_size=(2, 2), strides=None, padding='valid', mating_func=None, config=None, input_shape=None, mutable=False, **tf_kwargs)¶

Bases: nevopy.fixed_topology.layers.tf_layers.TensorFlowLayer

Wraps a TensorFlow 2D max pooling layer.

This is a simple wrapper for tf.keras.layers.MaxPool2D.

class nevopy.fixed_topology.layers.tf_layers.TensorFlowLayer(layer_type, mating_func=<function exchange_units_mating>, config=None, input_shape=None, mutable=True, **tf_kwargs)¶

Bases: nevopy.fixed_topology.layers.base_layer.BaseLayer

Wraps a TensorFlow layer.

This class wraps a TensorFlow layer, making it compatible with NEvoPy’s neuroevolutionary algorithms. It handles the mutation and reproduction of the TensorFlow layer.

In most cases, there is no need to create subclasses of this class. Doing that to frequently used types of layers, however, may be desirable, since it makes using those types of layers easier (see TFConv2DLayer and TFDenseLayer as examples).

When inheriting this class, you’ll usually do something like this:

class MyTFLayer(TensorFlowLayer):
    def __init__(self,
                 arg1, arg2,
                 activation="relu",
                 mating_func=mating.exchange_units_mating,
                 config=None,
                 input_shape=None,
                 mutable=True,
                 **tf_kwargs: Dict[str, Any]):
        super().__init__(
            layer_type=tf.keras.layers.SomeKerasLayer,
            **{k: v for k, v in locals().items()
               if k not in ["self", "tf_kwargs", "__class__"]},
            **tf_kwargs,
        )

Parameters

layer_type (Union[str, Type[tf.keras.layers.Layer]]) – A reference to the TensorFlow’s class that represents the layer (tf.keras.layers.Dense, for example). If it’s a string, the appropriate type will be inferred (note that it must be listed in TensorFlowLayer.KERAS_LAYERS).
mating_func (Optional[Callable[[BaseLayer, BaseLayer], BaseLayer]]) – Function that mates (sexual reproduction) two layers. It should receive two layers as input and return a new layer (the offspring). You can use one of the pre-built mating functions (see fixed_topology.layers.mating) or implement your own. If the layer is immutable, this parameter should receive None as argument.
config (Optional[FixedTopologyConfig]) – Settings being used in the current evolutionary session. If None, a config object must be assigned to the layer later on, before calling the methods that require it.
input_shape (Optional[Tuple[int, ..]]) – Shape of the data that will be processed by the layer. If None, an input shape for the layer must be manually specified later or be inferred from an input sample.
mutable (Optional[bool]) – Whether or not the layer can have its weights changed (mutation).
**tf_kwargs – Named arguments to be passed to the constructor of the TensorFlow layer.

KERAS_LAYERS = {'conv2D': <class 'tensorflow.python.keras.layers.convolutional.Conv2D'>, 'dense': <class 'tensorflow.python.keras.layers.core.Dense'>, 'flatten': <class 'tensorflow.python.keras.layers.core.Flatten'>, 'lstm': <class 'tensorflow.python.keras.layers.recurrent_v2.LSTM'>, 'max_pool_2D': <class 'tensorflow.python.keras.layers.pooling.MaxPooling2D'>, 'rnn': <class 'tensorflow.python.keras.layers.recurrent.RNN'>, 'simple_rnn': <class 'tensorflow.python.keras.layers.recurrent.SimpleRNN'>}¶

build(input_shape)¶

Wrapper for tf.keras.layers.Layer.build().

Return type: None

deep_copy()¶

Makes an exact/deep copy of the layer.

Return type: TensorFlowLayer
Returns: An exact/deep copy of the layer, including its weights and biases.

mate(other)¶

Mates two layers to produce a new layer (offspring).

Implements the sexual reproduction between a pair of layers. The new layer inherits information from both parents (not necessarily in an equal proportion)

Parameters: other (Any) – The second layer . If it’s not compatible for mating with the current layer (self), an exception will be raised.
Return type: TensorFlowLayer
Returns: A new layer (the offspring born from the sexual reproduction between the current layer and the layer passed as argument. If the layer is immutable, other is expected to be equal to self, so a deep copy (BaseLayer.deep_copy()) of the layer is returned.
Raises: IncompatibleLayersError – If the layer passed as argument to other is incompatible with the current layer (self).

mutate_weights(_test_info=None)¶

Randomly mutates the weights of the layer’s connections.

Each weight has a chance to be perturbed by a predefined amount or to be reset. The probabilities are obtained from the settings of the current evolutionary session.

If the layer is immutable, nothing happens (the layer’s weights remain unchanged).

Return type: None

process(x)¶

Feeds the given input(s) to the layer.

This is where the layer’s logic lives. If the layer hasn’t been built yet, it will be automatically built using the given input shape.

Parameters: x (Any) – The input(s) to be fed to the layer. Usually a NumPy ndarray or a TensorFlow tensor.
Return type: Tensor
Returns: The output of the layer. Usually a NumPy ndarray or a TensorFlow tensor.
Raises: InvalidInputError – If the shape of x doesn’t match the input shape expected by the layer.

random_copy()¶

Makes a random copy of the layer.

Return type: TensorFlowLayer
Returns: A new layer with the same topology of the current layer, but with newly initialized weights and biases. If the layer is immutable, a deep copy (BaseLayer.deep_copy()) of the layer is returned instead.

property tf_layer¶

The tf.keras.layers.Layer used internally.

Return type: Layer

property weights¶

The current weight matrices of the layer.

Wrapper for tf.keras.layers.Layer.get_weights().

The weights of a layer represent the state of the layer. This property returns the weight values associated with this layer as a list of Numpy arrays. In most cases, it’s a list containing the weights of the layer’s connections and the bias values (one for each neuron, generally).

Return type: List[ndarray]

Module contents¶

Neural network layers to be used with NEvoPy’s fixed-topology neuroevolutionary algorithms.