nevopy.utils.gym_utils package

Submodules

nevopy.utils.gym_utils.callbacks module

This module defines an interface for callbacks to be used with GymFitnessFunction.

class nevopy.utils.gym_utils.callbacks.BatchObsGymCallback

Bases: nevopy.utils.gym_utils.callbacks.GymCallback

Simple callback that expands the dimensions of the observations yielded by a gym.Env before feeding them to a genome.

Simply turns the observation into a batch of one item (the observation itself), so it can be fed to genomes that require batched inputs (like genomes that use TensorFlow, for example).

on_obs_processing(wrapped_obs)

Called right BEFORE the observation yielded by the environment is fed to the genome.

Changing the observation stored by the wrapper will have effects on the fitness function.

Subclasses should override this method for any actions to run.

Parameters

wrapped_obs (nevopy.utils.MutableWrapper[Any]) – Mutable wrapper around he observation yielded by the gym environment.

Return type

None

class nevopy.utils.gym_utils.callbacks.GymCallback

Bases: object

Interface for callbacks to be used with GymFitnessFunction.

Each of the callback’s method is called at a different point during the evaluation of a genome’s fitness by a GymFitnessFunction.

It’s not required for a subclass to implement all the methods of this class (you can implement only those that will be useful for your case).

on_action_chosen(wrapped_action)

Called right AFTER an action is chosen by the genome.

Changing the action stored by the wrapper will have effects on the fitness function.

Subclasses should override this method for any actions to run.

Parameters

wrapped_action (nevopy.utils.MutableWrapper[Any]) – Mutable wrapper around the action chosen by the genome.

Return type

None

on_env_built(env, genome)

Called right AFTER the gym environment is built.

This method is called right after the gym environment is built, i.e., right after a call to gym.make() is made.

Parameters

  • env (gym.Env) – The gym environment that’s going to be used by the fitness function.

  • genome (nevopy.BaseGenome) – The genome currently being evaluated by the fitness function.

Return type

None

on_env_close()

Called right BEFORE the environment is closed and the function returns the fitness of the genome.

Subclasses should override this method for any actions to run.

Return type

None

on_episode_start(current_eps, total_eps)

Called at the start of a new episode, before the env is reset.

Subclasses should override this method for any actions to run.

Parameters

  • current_eps (int) – Number of the current episode.

  • total_eps (int) – Total number of episodes to run during the current session.

Return type

None

on_obs_processing(wrapped_obs)

Called right BEFORE the observation yielded by the environment is fed to the genome.

Changing the observation stored by the wrapper will have effects on the fitness function.

Subclasses should override this method for any actions to run.

Parameters

wrapped_obs (nevopy.utils.MutableWrapper[Any]) – Mutable wrapper around he observation yielded by the gym environment.

Return type

None

on_step_start(current_step, max_steps)

Called at the start of a new step.

Subclasses should override this method for any actions to run.

Parameters

  • current_step (int) – Number of the current step.

  • max_steps (int) – Maximum number of steps allowed in each episode.

Return type

None

on_step_taken(obs, reward, done, info, total_reward, force_stop_eps)

Called right AFTER the environment’s step() method is called.

Subclasses should override this method for any actions to run.

Parameters

  • obs (Any) – The observation yielded by the environment.

  • reward (float) – The reward yielded by the environment.

  • done (bool) – Whether or not the episode has finished.

  • info (Dict[str, Any]) – Extra information yielded by the environment.

  • total_reward (float) – Total reward obtained by the genome so far.

  • force_stop_eps (nevopy.utils.MutableWrapper[bool]) – Setting the value on this wrapper to True will forcefully stop the current episode.

Return type

None

on_visualization()

Called right BEFORE the rendering of the environment occurs.

This method is only called when True is passed to the visualize parameter of GymFitnessFunction.__call__().

Subclasses should override this method for any actions to run.

Return type

None

nevopy.utils.gym_utils.fitness_function module

This module implements a generalizable fitness function that can be used with most gym environments.

class nevopy.utils.gym_utils.fitness_function.GymFitnessFunction(make_env, env_renderer=None, callbacks=None, default_num_episodes=1, default_max_steps=None, num_obs_skip=0)

Bases: object

Wrapper for a fitness function to be used with gym.

This utility class implements a generalizable fitness function compatible with different gym environments.

Parameters

  • make_env (Callable[[], gym.Env]) – Callable that creates the environment to be used. It should receive no arguments and return an instance of gym.Env.

  • env_renderer (Optional[GymRenderer]) – Instance of GymRenderer (or a subclass) to be used to render the environment. By default, a new instance of GymRenderer is created (default rendering of the environment).

  • callbacks (Optional[List[GymCallback]]) – List with callbacks to be called at different stages of the evaluation of the genome’s fitness.

  • default_num_episodes (int) – Default number of episodes ran in each call to the fitness function. This can be overridden during the call to the fitness function.

  • default_max_steps (Optional[int]) – Default maximum number of steps allowed in each episode. By default, there is no limit to the number of steps. This can be overridden during the call to the fitness function.

  • num_obs_skip (int) – Number of observations to be skipped during an episode. As an example, consider this value is set to 3. In this case, for each sequence of 4 observations yielded by the environment, only the 1st one will be fed to the genome. When, during a step, no observation is fed to the genome, the genome’s last output is used to advance the environment’s state.

env_renderer

Instance of GymRenderer (or a subclass) to be used to render the environment.

Type

GymRenderer

callbacks

List with callbacks to be called at different stages of the evaluation of the genome’s fitness.

Type

List[GymCallback]

num_obs_skip

Number of observations to be skipped during an episode.

Type

int

nevopy.utils.gym_utils.renderers module

This module implements the entities responsible for rendering a gym.Env during the evaluation of a genome’s fitness by a GymFitnessFunction.

class nevopy.utils.gym_utils.renderers.GymRenderer(fps=45)

Bases: object

Defines the entity responsible for rendering a gym.Env during the evaluation of a genome’s fitness by a GymFitnessFunction.

Parameters

fps (int) – Frames per second.

fps

Frames per second.

Type

int

flush()

Flushes the internal buffers of the renderer.

Doesn’t do anything by default. Subclasses should override this method in order for any action to occur.

Return type

None

render(env, genome)

Renders the environment in “human mode”.

Parameters

  • env (gym.Env) – Environment to be rendered.

  • genome (BaseGenome) – Genome currently being evaluated.

Return type

None

class nevopy.utils.gym_utils.renderers.NeatActivationsGymRenderer(out_path='./gym_videos', fps=30, play_video=True, **kwargs)

Bases: nevopy.utils.gym_utils.renderers.GymRenderer

Gym env renderer that renders a NEAT genome’s neural network while the genome is interacting with the environment.

Three videos will be generated: one containing the recording of the genome’s interactions with the environment, another containing the genome’s neural network states during the interactions and another containing the concatenation of the two previously videos, side by side.

Note

Compatible with NeatGenome only!

Note

This renderer requires you to have the opencv-python and scikit-video packages installed. You can install them using pip. You’ll also need FFmpeg.

Parameters

  • out_path (str) – Path to the output directory.

  • fps (int) – Frames per second of the generated videos.

  • play_video (bool) – If True, the concatenated videos will be automatically played after the rendering is done.

  • **kwargs (Dict[str, Any]) – Named arguments to be passed to genome.visualize_activations().

flush()

Generates the videos from the images in the cache, closes the necessary resources and clears the image cache.

Return type

None

static play_video(video_file, fps)

Plays a video from a file. ?????

Return type

None

render(env, genome)

Renders the environment in “human mode”.

Parameters

  • env (gym.Env) – Environment to be rendered.

  • genome (BaseGenome) – Genome currently being evaluated.

Return type

None

Module contents

Exposes the core functionalities of gym_utils.