Project Columbus

Project Columbus is a framework for trivial 2D OpenAI Gym environments that are supposed to test a agents ability to solve tasks that require different forms of exploration effectively and efficiently.

Installation

(If you want to install Columbus as a dependency for metastable-baselines, activate (source) the venv from metastable-baselines before running this command.)

pip install -e .

env.py

Contains the ColumbusEnv. There exist two ways to implement new envs:

• Subclassing ColumbusEnv and expanding init and overriding setup.
• Using the ColumbusConfigDefined with a desired configuration. This makes configuring ColumbusEnvs via ClusterWorks2-configs possible. (See ColumbusConfigDefinedExample.md for an example of how the parameters are supposed to look like (uses yaml format), I don't have to to write a better documentation right now...)

Some caveats / infos

• If you want to render to a window (pygame-gui) call render with mode='human'
• If you want visualize the covariance you have supply the cholesky-decomp of the cov-matrix to render
• If you want to render into a mp4, you have to call render with a mode!='human' and assemble/encode the returned frames yourself into a mp4/webm/...
• Even while the agent plays, some keyboard-inputs are possible (to test the agents reaction to situations he would never enter by itself. Look at _handle_user_input in env.py for avaible keys)

entities.py

Contains all implemented entities (e.g. the Agent, Rewards and Enemies)

Some caveats

• Support for non spherical entities (rectangles) is very new. There might be bugs that I have not yet found

observables.py

Contains all 'oberservables'. These are attached to envs to define what kind of output is given to the agent. This way environments can be designed independently from the observation machanism that is used by the agent to play it.

Some caveats

• CNNObservable seems to be broken currently. (Fixing it is also no priority for me)

humanPlayer.py

Allows environments to be played by a human using mouse input.

Some caveats

• Does not yet work for ColumbusConfigDefined...