Made path_decay configurable

rendering

.gitignore

@@ -2,3 +2,4 @@ __pychache__
 *.pyc
 *.pyo
 *.egg-info
+.venv

LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2022 Dominik Roth
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

README.md

@@ -1,17 +1,57 @@
 # Project Columbus
 
+<p align='center'>
+  <img src='./icon.svg'>
+</p>
+
 Project Columbus is a framework for trivial 2D OpenAI Gym environments that are supposed to test a agents ability to solve tasks that require different forms of exploration effectively and efficiently.
 
-![Screenshot](./img_README.png)
+## Installation
+
+(If you want to install Columbus as a dependency for metastable-baselines, activate (source) the venv from metastable-baselines before running this command.)
+
+```
+pip install -e .
+```
+
+## Overview
+
+Columbus.pdf contains a overview of columbus.
+
+## Layout of the Repo
 
 ### env.py
-Contains the ColumbusEnv. New envs are implemented by subclassing ColumbusEnv and expanding _init_ and overriding _setup_.
+
+![Screenshot](./img_README.png)  
+Contains the ColumbusEnv.
+There exist two ways to implement new envs:
+
+- Subclassing ColumbusEnv and expanding _init_ and overriding _setup_.
+- Using the ColumbusConfigDefined with a desired configuration. This makes configuring ColumbusEnvs via ClusterWorks2-configs possible. (See configs/example.yaml for an example of how the parameters are supposed to look like (uses yaml format)
+- We now support using units (px, em, ct) in config files, examples can be found in configs/Example_Units.yaml
+- The environments used in my thesis can also be found in configs/
+
+##### Some caveats / infos
+
+- If you want to render to a window (pygame-gui) call render with mode='human'
+- If you want visualize the covariance you have to supply the cholesky-decomp of the cov-matrix to render
+- If you want to render into a mp4, you have to call render with a mode!='human' and assemble/encode the returned frames yourself into a mp4/webm/...
+- Even while the agent plays, some keyboard-inputs are possible (to test the agents reaction to situations he would never enter by itself. Look at \_handle_user_input in env.py for avaible keys)
+- The sampling-rate of the physics engine is bound to the frame-rate of the rendering engine (1:1). This means too low fps / too fast agents / too thin barriers will lead to the agent tunneling through barriers. You can fix this by setting a higher agent-drag (which decreases the maximum speed) or making barriers thicker. A feature allowing the physics engine to sample multiple smaller steps within a single rendering step could be added in the future.
 
 ### entities.py
+
 Contains all implemented entities (e.g. the Agent, Rewards and Enemies)
 
 ### observables.py
+
 Contains all 'oberservables'. These are attached to envs to define what kind of output is given to the agent. This way environments can be designed independently from the observation machanism that is used by the agent to play it.
 
+##### Some caveats
+
+- CNNObservable seems to be broken currently. (Fixing it is also no priority for me)
+- RayObservable is using a naive ray-marching (basicaly just line-sweeping). For large amounts of rays this turn out to be the computational bottleneck of the environment. Switching to a more efficient algorithm (based on euclidean formulars and line intersects) would be possible in the future...
+
 ### humanPlayer.py
-Allows environments to be played by a human using mouse input.
+
+Allows environments to be played by a human using mouse input. Now even works for ColumbusConfigDefined.

columbus/__init__.py

@@ -1 +1 @@
-# emtpy
+from columbus import env

columbus/entities.py

@@ -3,28 +3,61 @@ import math
 
 
 class Entity(object):
+    def __call__(cls, *args, **kwargs):
+        obj = type.__call__(cls, *args, **kwargs)
+        obj.__post_init__()
+        return obj
+
     def __init__(self, env):
+        self.shape = None
         self.env = env
         self.pos = (env.random(), env.random())
+        self.last_pos = None
         self.speed = (0, 0)
         self.acc = (0, 0)
         self.drag = 0
-        self.radius = 10
         self.col = (255, 255, 255)
-        self.shape = 'circle'
+        self.solid = False
+        self.movable = False  # False = Non movable, True = Movable, x>1: lighter movable
+        self.void_collidable = False
+        self.elasticity = 1
+        self.collision_changes_speed = self.env.controll_type == 'ACC'
+        self.collision_elasticity = self.env.default_collision_elasticity
+        self._crash_list = []
+        self._coll_add_pushback = 0
+        self.crash_conservation_of_energy = True
+        self.draw_path = False
+        self.draw_path_col = [int(c/5) for c in self.col]
+        self.draw_path_width = 2
+        self.draw_path_harm = False
+        self.draw_path_harm_col = [c for c in self.draw_path_col]
+        self.draw_path_harm_col[0] += int(255/3)
+        self.min_speed = 0
+        self.max_speed = math.inf
+
+    def __post_init__(self):
+        pass
 
     def physics_step(self):
         x, y = self.pos
         vx, vy = self.speed
         ax, ay = self.acc
         vx, vy = vx+ax*self.env.acc_fac,  vy+ay*self.env.acc_fac
+        speeds = math.sqrt(vx**2 + vy**2)
+        if speeds < self.min_speed:
+            vx, vy = vx/speeds*self.min_speed, vy/speeds*self.min_speed
+        if speeds > self.max_speed:
+            vx, vy = vx/speeds*self.max_speed, vy/speeds*self.max_speed
         x, y = x+vx*self.env.speed_fac, y+vy*self.env.speed_fac
-        if x > 1 or x < 0:
-            x = min(max(x, 0), 1)
-            vx = 0
-        if y > 1 or y < 0:
-            y = min(max(y, 0), 1)
-            vy = 0
+        if not self.env.torus_topology and self.void_collidable:
+            if x > 1 or x < 0:
+                x, y, vx, vy = self.calc_void_collision(x < 0, x, y, vx, vy)
+            if y > 1 or y < 0:
+                x, y, vx, vy = self.calc_void_collision(
+                    2 + (x < 0), x, y, vx, vy)
+        else:
+            x = x % 1
+            y = y % 1
         self.speed = vx/(1+self.drag), vy/(1+self.drag)
         self.pos = x, y
 
@@ -34,26 +67,281 @@ class Entity(object):
     def step(self):
         self.controll_step()
         self.physics_step()
+        self._crash_list = []
 
     def draw(self):
-        x, y = self.pos
-        pygame.draw.circle(self.env.surface, self.col,
-                           (x*self.env.width, y*self.env.height), self.radius, width=0)
+        self._draw_path()
 
-    def on_collision(self, other):
+    def _draw_path(self):
+        if self.draw_path and self.last_pos:
+            col = self.draw_path_col
+            if self.draw_path_harm:
+                if self.env.gotHarm:
+                    col = self.draw_path_harm_col
+            pygame.draw.line(self.env.path_overlay, col,
+                             (self.last_pos[0]*self.env.width, self.last_pos[1]*self.env.height), (self.pos[0]*self.env.width, self.pos[1]*self.env.height), self.draw_path_width)
+            pygame.draw.circle(self.env.path_overlay, col,
+                               (self.pos[0]*self.env.width, self.pos[1]*self.env.height), max(0, self.draw_path_width/2-3))
+        self.last_pos = self.pos[0], self.pos[1]
+
+    def on_collision(self, other, depth):
+        if self.solid and other.solid:
+            if self.movable:
+                self.on_crash(other, depth)
+
+    def on_crash(self, other, depth):
+        if other in self._crash_list:
+            return
+        self._crash_list.append(other)
+        force_dir = self._get_crash_force_dir(other)
+        #print(force_dir, depth)
+        force_dir_len = math.sqrt(force_dir[0]**2+force_dir[1]**2)
+        if force_dir_len == 0:
+            return
+        force_dir = force_dir[0]/force_dir_len, force_dir[1]/force_dir_len
+        if not self.env.torus_topology:
+            if self == self.env.agent:
+                agent = self
+            elif other == self.env.agent:
+                agent = other
+            else:
+                agent = None
+            if agent:
+                if agent.pos[0] > 0.99 or agent.pos[0] < 0.01:
+                    force_dir = force_dir[0], force_dir[1] * 2
+                if agent.pos[1] > 0.99 or agent.pos[1] < 0.01:
+                    force_dir = force_dir[0] * 2, force_dir[1]
+        depth *= 1.0*self.movable/(self.movable + other.movable)/2
+        depth /= other.elasticity
+        force_vec = force_dir[0]*depth/self.env.width, \
+            force_dir[1]*depth/self.env.height
+        if sum([abs(f) for f in force_vec]) > 0.005:
+            self.pos = self.pos[0] + force_vec[0], self.pos[1] + force_vec[1]
+            if self._coll_add_pushback:
+                self.pos = self.pos[0] - self.env.inp[0]*self._coll_add_pushback * \
+                    self.env.speed_fac, self.pos[1] - self.env.inp[1] * \
+                    self._coll_add_pushback*self.env.speed_fac
+        if self.collision_changes_speed:
+            oldspeed = math.sqrt(self.speed[0]**2+self.speed[1]**2)
+            self.speed = self.speed[0] + \
+                force_vec[0]*self.collision_elasticity/self.env.speed_fac, self.speed[1] + \
+                force_vec[1]*self.collision_elasticity/self.env.speed_fac
+            newspeed = math.sqrt(self.speed[0]**2+self.speed[1]**2)
+            if self.crash_conservation_of_energy and newspeed > oldspeed*1.1:
+                self.speed = self.speed[0]/newspeed*1.1 * \
+                    oldspeed, self.speed[1]/newspeed*oldspeed*1.1
+
+    def _get_crash_force_dir(self, other):
+        if 1 == 1:  # linter hack
+            raise Exception(
+                '[!] No collision-logic implemented for shape"'+str(self.shape)+'"')
+
+    def on_collect(self, other):
         pass
 
+    def on_collected(self):
+        pass
+
+    def calc_void_collision(self, dir, x, y, vx, vy):
+        if dir < 2:
+            x = min(max(x, 0), 1)
+            vx = -vx*self.collision_elasticity*0.5*self.collision_changes_speed
+        else:
+            y = min(max(y, 0), 1)
+            vy = -vy*self.collision_elasticity*0.5*self.collision_changes_speed
+        return x, y, vx, vy
+
     def kill(self):
         self.env.kill_entity(self)
 
+    def getQuasiRadius(self):
+        raise Exception()
 
-class Agent(Entity):
+    def getTop(self):
+        raise Exception()
+
+    def getBottom(self):
+        raise Exception()
+
+    def getLeft(self):
+        raise Exception()
+
+    def getRight(self):
+        raise Exception()
+
+    def getCenter(self):
+        raise Exception()
+
+
+class CircularEntity(Entity):
+    def __init__(self, env):
+        super().__init__(env)
+        self.shape = 'circle'
+        self.radius = 10
+
+    def draw(self):
+        super().draw()
+        x, y = self.pos
+        pygame.draw.circle(self.env.surface, self.col,
+                           (x*self.env.width, y*self.env.height), self.radius, width=0)
+
+    def _get_crash_force_dir(self, other):
+        if other.shape == 'circle':
+            return self.pos[0] - other.pos[0], self.pos[1] - other.pos[1]
+        elif other.shape == 'rect':
+            pad = 0
+            edge_size = min(self.radius, min(
+                other.width/3, other.height/3)) + 1
+
+            x, y = self.pos
+            x, y = x*self.env.height, y*self.env.width
+            left, top = x - self.radius + pad, y - self.radius + pad
+            right, bottom = x + self.radius - pad, y + self.radius - pad
+            lrcenter, tbcenter = x, y
+
+            ox, oy = other.pos
+            ox, oy = ox*self.env.height, oy*self.env.width
+            oleft, otop = ox + pad, oy + pad
+            oright, obottom = ox + other.width - pad, oy + other.height - pad
+            olrcenter, otbcenter = ox + other.width/2, oy + other.height/2
+
+            lr, tb = 0, 0
+
+            if otop < bottom and obottom > bottom:
+                # col from top
+                tb = otop - bottom
+                #print('t', tb)
+            elif top < obottom and top > otop:
+                # col from bottom
+                tb = - top + obottom
+                #print('b', tb)
+
+            if right > oleft and right < oright:
+                # col from left
+                lr = oleft - right
+                #print('l', lr)
+            elif left < oright and left > oleft:
+                # col from right
+                lr = - left + oright
+                #print('r', lr)
+
+            if lr != 0 and tb != 0:
+                if abs(abs(tb) - abs(lr)) < edge_size:
+                    if abs(tb) < abs(lr):
+                        return lr/5, tb
+                    else:
+                        return lr, tb/5
+                if abs(tb) < abs(lr):
+                    return 0, tb
+                else:
+                    return lr, 0
+
+            return 0, 0
+        else:
+            raise Exception(
+                '[!] Shape "circle" does not know how to collide with shape "'+str(other.shape)+'"')
+
+    def getQuasiRadius(self):
+        return self.radius
+
+    def getTop(self):
+        return self.pos[1]*self.env.height - self.radius
+
+    def getBottom(self):
+        return self.pos[1]*self.env.height + self.radius
+
+    def getLeft(self):
+        return self.pos[0]*self.env.width - self.radius
+
+    def getRight(self):
+        return self.pos[0]*self.env.width + self.radius
+
+    def getCenter(self):
+        return self.pos[0]*self.env.width, self.pos[1]*self.env.height
+
+
+class RectangularEntity(Entity):
+    def __init__(self, env):
+        super().__init__(env)
+        self.shape = 'rect'
+        self.width = 10
+        self.height = 10
+
+    def draw(self):
+        super().draw()
+        x, y = self.pos
+        rect = pygame.Rect(x*self.env.width, y *
+                           self.env.width, self.width, self.height)
+        pygame.draw.rect(self.env.surface, self.col,
+                         rect, width=0)
+
+    def _get_crash_force_dir(self, other):
+        raise Exception(
+            '[!] Collisions in this direction not implemented for shape "rectangle"')
+
+    def physics_step(self):
+        x, y = self.pos
+        vx, vy = self.speed
+        ax, ay = self.acc
+        vx, vy = vx+ax*self.env.acc_fac,  vy+ay*self.env.acc_fac
+        speeds = math.sqrt(vx**2 + vy**2)
+        if speeds < self.min_speed:
+            vx, vy = vx/speeds*self.min_speed, vy/speeds*self.min_speed
+        if speeds > self.max_speed:
+            vx, vy = vx/speeds*self.max_speed, vy/speeds*self.max_speed
+        x, y = x+vx*self.env.speed_fac, y+vy*self.env.speed_fac
+        if not self.env.torus_topology and self.void_collidable:
+            if x+(self.width/self.env.width) > 1 or x < 0:
+                if x < 0:
+                    x, y, vx, vy = self.calc_void_collision(
+                        x < 0, x, y, vx, vy)
+                else:
+                    x, y, vx, vy = self.calc_void_collision(
+                        x < 0, x+(self.width/self.env.width), y, vx, vy)
+                    x -= (self.width/self.env.width)
+            if y+(self.height/self.env.height) > 1 or y < 0:
+                if y < 0:
+                    x, y, vx, vy = self.calc_void_collision(
+                        2 + (x < 0), x, y, vx, vy)
+                else:
+                    x, y, vx, vy = self.calc_void_collision(
+                        2 + (x < 0), x, y+(self.height/self.env.height), vx, vy)
+                    y -= (self.height/self.env.height)
+        else:
+            x = x % 1
+            y = y % 1
+        self.speed = vx/(1+self.drag), vy/(1+self.drag)
+        self.pos = x, y
+
+    def getQuasiRadius(self):
+        return self.width + self.height
+
+    def getTop(self):
+        return self.pos[1]*self.env.height
+
+    def getBottom(self):
+        return self.pos[1]*self.env.height + self.height
+
+    def getLeft(self):
+        return self.pos[0]*self.env.width
+
+    def getRight(self):
+        return self.pos[0]*self.env.width*self.env.height + self.width
+
+    def getCenter(self):
+        return self.pos[0]*self.env.width+self.width/2, self.pos[1]*self.env.height+self.height/2
+
+
+class Agent(CircularEntity):
     def __init__(self, env):
         super(Agent, self).__init__(env)
         self.pos = (0.5, 0.5)
         self.col = (0, 0, 255)
         self.drag = self.env.agent_drag
         self.controll_type = self.env.controll_type
+        self.solid = True
+        self.movable = True
+        self.void_collidable = True
 
     def controll_step(self):
         self._read_input()
@@ -68,13 +356,39 @@ class Agent(Entity):
             raise Exception('Unsupported controll_type')
 
 
+# Does not work! Don't use!
+class PongAgent(RectangularEntity):
+    def __init__(self, env):
+        super(PongAgent, self).__init__(env)
+        self.pos = (0.5, 0.5)
+        self.col = (0, 0, 255)
+        self.drag = self.env.agent_drag
+        self.controll_type = self.env.controll_type
+        self.solid = True
+        self.movable = True
+
+    def controll_step(self):
+        self._read_input()
+        self.env.check_collisions_for(self)
+
+    def _read_input(self):
+        if self.controll_type == 'SPEED':
+            self.speed = 0, self.env.inp[1] - 0.5
+        elif self.controll_type == 'ACC':
+            self.acc = 0, self.env.inp[1] - 0.5
+        else:
+            raise Exception('Unsupported controll_type')
+
+
 class Enemy(Entity):
     def __init__(self, env):
         super(Enemy, self).__init__(env)
         self.col = (255, 0, 0)
         self.damage = 100
+        self.radiateDamage = True
 
-    def on_collision(self, other):
+    def on_collision(self, other, depth):
+        super().on_collision(other, depth)
         if isinstance(other, Agent):
             self.env.new_reward -= self.damage
 
@@ -82,14 +396,21 @@ class Enemy(Entity):
 class Barrier(Enemy):
     def __init__(self, env):
         super(Barrier, self).__init__(env)
+        self.solid = True
+        self.movable = False
 
 
-class CircleBarrier(Barrier):
+class CircleBarrier(Barrier, CircularEntity):
     def __init__(self, env):
         super(CircleBarrier, self).__init__(env)
 
 
-class Chaser(Enemy):
+class RectBarrier(Barrier, RectangularEntity):
+    def __init__(self, env):
+        super().__init__(env)
+
+
+class Chaser(Enemy, CircularEntity):
     def __init__(self, env):
         super(Chaser, self).__init__(env)
         self.target = self.env.agent
@@ -129,21 +450,25 @@ class FlyingChaser(Chaser):
         self.acc = arrow[0] * self.chase_acc, arrow[1] * self.chase_acc
 
 
-class Reward(Entity):
+class Collectable(CircularEntity):
     def __init__(self, env):
-        super(Reward, self).__init__(env)
-        self.col = (0, 255, 0)
+        super(Collectable, self).__init__(env)
         self.avaible = True
         self.enforce_not_on_barrier = False
-        self.reward = 1
+        self.reward = 10
+        self.collectors = []
 
-    def on_collision(self, other):
-        if isinstance(other, Agent):
-            self.on_collect()
-        elif isinstance(other, Barrier):
+    def on_collision(self, other, depth):
+        super().on_collision(other, depth)
+        if isinstance(other, Barrier):
             self.on_barrier_collision()
+        else:
+            for Col in self.collectors:
+                if isinstance(other, Col):
+                    other.on_collect(self)
+                    self.on_collected()
 
-    def on_collect(self):
+    def on_collected(self):
         self.env.new_reward += self.reward
 
     def on_barrier_collision(self):
@@ -152,12 +477,50 @@ class Reward(Entity):
             self.env.check_collisions_for(self)
 
 
+class RectCollectable(RectangularEntity):
+    def __init__(self, env):
+        super(RectCollectable, self).__init__(env)
+        self.avaible = True
+        self.enforce_not_on_barrier = False
+        self.reward = 10
+        self.collectors = []
+
+    def on_collision(self, other, depth):
+        super().on_collision(other, depth)
+        if isinstance(other, Barrier):
+            self.on_barrier_collision()
+        else:
+            for Col in self.collectors:
+                if isinstance(other, Col):
+                    other.on_collect(self)
+                    self.on_collected()
+
+    def on_collected(self):
+        self.env.new_reward += self.reward
+
+    def on_barrier_collision(self):
+        if self.enforce_not_on_barrier:
+            self.pos = (self.env.random(), self.env.random())
+            self.env.check_collisions_for(self)
+
+
+class Reward(Collectable):
+    def __init__(self, env):
+        super(Reward, self).__init__(env)
+        self.col = (0, 255, 0)
+        self.reward = 10
+        self.collectors = [Agent]
+
+
 class OnceReward(Reward):
     def __init__(self, env):
         super(OnceReward, self).__init__(env)
-        self.reward = 100
+        self.reward = 500
+
+    def on_collected(self):
+        # Force rerender of value func (even in static envs)
+        self.env._invalidate_value_map()
 
-    def on_collect(self):
         self.env.new_abs_reward += self.reward
         self.kill()
 
@@ -168,12 +531,51 @@ class TeleportingReward(OnceReward):
         self.enforce_not_on_barrier = True
         self.env.check_collisions_for(self)
 
-    def on_collect(self):
+    def on_collected(self):
+        # Force rerender of value func (even in static envs)
+        self.env._invalidate_value_map()
+
         self.env.new_abs_reward += self.reward
         self.pos = (self.env.random(), self.env.random())
         self.env.check_collisions_for(self)
 
 
+class LoopReward(OnceReward):
+    def __init__(self, env):
+        super().__init__(env)
+        self.loop = [[0.25, 0.5], [0.75, 0.5]]
+        self.state = 0
+        self.jump_to_state()
+        self.barrier_physics = False
+
+    def jump_to_state(self):
+        # Force rerender of value func (even in static envs)
+        self.env._invalidate_value_map()
+
+        pos_vec = [v for v in self.loop[self.state]]
+        if len(pos_vec) == 4:
+            pos_vec = pos_vec[0] + pos_vec[2] * \
+                (self.env.random()-0.5), pos_vec[1] + \
+                pos_vec[3]*(self.env.random()-0.5)
+        self.pos = pos_vec
+
+    def next_state(self):
+        self.state = (self.state + 1) % len(self.loop)
+
+    def jump_next(self):
+        self.next_state()
+        self.jump_to_state()
+
+    def on_collected(self):
+        self.env.new_abs_reward += self.reward
+        self.jump_next()
+
+    def physics_step(self):
+        if self.barrier_physics:
+            self.env.check_collisions_for(self)
+        super().physics_step()
+
+
 class TimeoutReward(OnceReward):
     def __init__(self, env):
         super(TimeoutReward, self).__init__(env)
@@ -188,13 +590,96 @@ class TimeoutReward(OnceReward):
         else:
             self.col = (50, 100, 50)
 
-    def on_collect(self):
+    def on_collected(self):
         if self.avaible:
+            # Force rerender of value func (even in static envs)
+            self.env._invalidate_value_map()
+
             self.env.new_abs_reward += self.reward
             self.set_avaible(False)
             self.env.timers.append((self.timeout, self.set_avaible, True))
 
 
+class Ball(CircularEntity):
+    def __init__(self, env):
+        super(Ball, self).__init__(env)
+        self.col = (255, 128, 0)
+        self.drag = 0.0025
+        self.solid = True
+        self.movable = 10
+        self.elasticity = 1
+        self.collision_changes_speed = True
+        self.wall_reflect_damping = 0.1
+
+    def calc_void_collision(self, dir, x, y, vx, vy):
+        if dir < 2:
+            x = min(max(x, 0), 1)
+            vx = -(vx/(1+self.wall_reflect_damping))
+        else:
+            y = min(max(y, 0), 1)
+            vy = -(vy/(1+self.wall_reflect_damping))
+        return x, y, vx, vy
+
+    def physics_step(self):
+        self.env.check_collisions_for(self)
+        super().physics_step()
+
+
+class Goal(Collectable):
+    def __init__(self, env):
+        super(Goal, self).__init__(env)
+        self.col = (0, 200, 0)
+        self.reward = 500
+        self.radius = 20
+        self.collectors = [Ball]
+
+
+class RectGoal(RectCollectable):
+    def __init__(self, env):
+        super(RectGoal, self).__init__(env)
+        self.col = (0, 200, 0)
+        self.reward = 500
+        self.collectors = [Ball]
+
+
+class TeleportingGoal(Goal):
+    def __init__(self, env):
+        super(TeleportingGoal, self).__init__(env)
+        self.enforce_not_on_barrier = True
+        self.env.check_collisions_for(self)
+
+    def on_collected(self):
+        # Force rerender of value func (even in static envs)
+        self.env._invalidate_value_map()
+
+        self.env.new_abs_reward += self.reward
+        self.pos = (self.env.random(), self.env.random())
+        self.env.check_collisions_for(self)
+
+
+class FootballPlayer():
+    def __init__(self, env, target):
+        super(FootballPlayer, self).__init__(env)
+        self.col = (200, 0, 100)
+        self.target = target
+        self.solid = True
+        self.movable = 1
+        self.elasticity = 1
+
+
+class WalkingFootballPlayer(FootballPlayer, WalkingChaser):
+    def __init__(self, env, target):
+        super(WalkingFootballPlayer, self).__init__(env, target)
+        self.target = target
+
+
+class FlyingFootballPlayer(FootballPlayer, FlyingChaser):
+    def __init__(self, env, target):
+        super(FlyingFootballPlayer, self).__init__(env, target)
+
+# Not a real entity. Is used in the config of RayObserver to reference the outer boundary of the environment.
+
+
 class Void():
     def __init__(self, env):
         self.col = (50, 50, 50)

columbus/gym_binding.py

@@ -1,13 +0,0 @@
-from gym.envs.registration import register
-from env import *
-
-def register():
-    register(
-        # unique identifier for the env `name-version`
-        id="Columbus-Test317-v0",
-        # path to the class for creating the env
-        # Note: entry_point also accept a class as input (and not only a string)
-        entry_point=env.ColumbusEnv,
-        # Max number of steps per episode, using a `TimeLimitWrapper`
-        max_episode_steps=500,
-    )

columbus/humanPlayer.py

@@ -1,34 +1,92 @@
+import torch as th
 from time import sleep, time
-from env import *
 import numpy as np
 import pygame
+import yaml
 
-from observables import Observable, CnnObservable
+from columbus import env
+from columbus.observables import Observable, CnnObservable
 
 
 def main():
-    env = ColumbusTest3_1()
-    env = ColumbusTestRay(hide_map=True)
-    env.start_pos = [0.6, 0.3]
-    playEnv(env)
+    env = chooseEnv()
+    while True:
+        playEnv(env)
+        input('<again?>')
     env.close()
 
 
+def getAvaibleEnvs():
+    # kinda hacky... idk
+    strs = dir(env)
+    for s in strs:
+        if s.startswith('Columbus') and s != 'ColumbusEnv':
+            yield getattr(env, s)
+
+
+def loadConfigDefinedEnv(EnvClass):
+    p = input('[Path to config> ')
+    with open(p, 'r') as f:
+        docs = list([d for d in yaml.safe_load_all(
+            f) if d and 'name' in d and d['name'] not in ['SLURM']])
+    for i, doc in enumerate(docs):
+        name = doc['name']
+        print('['+str(i)+'] '+name)
+    ds = int(input('[0]> ') or '0')
+    doc = docs[ds]
+    cur = doc
+    path = 'params.task.env_args'
+    p = path.split('.')
+    while True:
+        try:
+            if len(p) == 0:
+                break
+            key = p.pop(0)
+            print(key)
+            cur = cur[key]
+        except Exception as e:
+            print('Unable to find key "'+key+'"')
+            path = input('[Path> ')
+    print(cur)
+    return EnvClass(fps=30, **cur)
+
+
+def chooseEnv():
+    envs = list(getAvaibleEnvs())
+    for i, Env in enumerate(envs):
+        print('['+str(i)+'] '+Env.__name__)
+    while True:
+        inp = input('[#> ')
+        try:
+            i = int(inp)
+        except:
+            print('[!] You have to enter the number...')
+        if i < 0 or i >= len(envs):
+            print(
+                '[!] That is a number, but not one that makes sense in this context...')
+        if envs[i] in [env.ColumbusConfigDefined]:
+            return loadConfigDefinedEnv(envs[i])
+        Env = envs[i]
+        return Env(fps=30)
+
+
+def value_func(obs):
+    return obs[:, 0]
+    # return th.rand(obs.shape[0])-0.5
+
+
 def playEnv(env):
-    env.reset()
     done = False
+    env.reset()
     while not done:
         t1 = time()
+        # env.render(value_func=value_func)
         env.render()
         pos = (0.5, 0.5)
-        for event in pygame.event.get():
-            pass
-            # if event.type == pygame.MOUSEBUTTONDOWN:
-            #    pos = pygame.mouse.get_pos()
-            #    print(pos)
         pos = pygame.mouse.get_pos()
         pos = (min(max((pos[0]-env.joystick_offset[0]-20)/60, 0), 1),
                min(max((pos[1]-env.joystick_offset[1]-20)/60, 0), 1))
+        pos = pos[0]*2-1, pos[1]*2-1
         obs, rew, done, info = env.step(np.array(pos, dtype=np.float32))
         print('Reward: '+str(rew))
         print('Score: '+str(info))

columbus/observables.py

@@ -3,29 +3,33 @@ import numpy as np
 import pygame
 import math
 from columbus import entities
+import torch as th
 
 
 class Observable():
     def __init__(self):
         self.obs = None
-        pass
 
     def _set_env(self, env):
         self.env = env
 
-    def get_observation_space():
+    def get_observation_space(self):
         print("[!] Using dummyObservable. Env won't output anything")
-        return spaces.Box(low=0, high=255,
-                          shape=(1,), dtype=np.uint8)
+        return spaces.Box(low=0, high=1,
+                          shape=(1,), dtype=np.float64)
 
     def get_observation(self):
-        return False
+        return np.array([0])
 
     def draw(self):
         pass
 
+    def reset(self):
+        pass
+
 
 class CnnObservable(Observable):
+    # Currently broken...
     def __init__(self, in_width=256, in_height=256, out_width=32, out_height=32, draw_width=128, draw_height=128, smooth_scaling=True):
         super(CnnObservable, self).__init__()
         self.in_width = in_width
@@ -41,11 +45,11 @@ class CnnObservable(Observable):
 
     def get_observation_space(self):
         return spaces.Box(low=0, high=255,
-                          shape=(self.out_width, self.out_height, 3), dtype=np.uint8)
+                          shape=(self.out_width, self.out_height, 3), dtype=np.float64)
 
     def get_observation(self):
         if not self.env._rendered:
-            self.env.render(dont_show=True)
+            self.env.render(mode='internal', dont_show=False)
         self.env._ensure_surface()
         x, y = self.env.agent.pos[0]*self.env.width - self.in_width / \
             2, self.env.agent.pos[1]*self.env.height - self.in_height/2
@@ -57,7 +61,11 @@ class CnnObservable(Observable):
         rect = pygame.Rect(cx, cy, cw, ch)
         snap = self.env.surface.subsurface(rect)
         self.snap = pygame.Surface((self.in_width, self.in_height))
-        pygame.draw.rect(self.snap, (50, 50, 50),
+        if self.env.void_barrier:
+            col = (255, 0, 0)
+        else:
+            col = (50, 50, 50)
+        pygame.draw.rect(self.snap, col,
                          pygame.Rect(0, 0, self.in_width, self.in_height))
         self.snap.blit(snap, (cx - x, cy - y))
         self.obs = self.scaler(
@@ -82,18 +90,19 @@ def _clip(num, lower, upper):
 
 
 class RayObservable(Observable):
-    def __init__(self, num_rays=24, chans=[entities.Enemy, entities.Reward, entities.Void], ray_len=256):
+    def __init__(self, num_rays=16, chans=[entities.Enemy, entities.Reward], ray_len=256, num_steps=64, include_rand=False):
         super(RayObservable, self).__init__()
         self.num_rays = num_rays
         self.chans = chans
         self.num_chans = len(chans)
         self.ray_len = ray_len
-        self.num_steps = 32  # max = 255
+        self.num_steps = num_steps  # max = 255
         self.occlusion = True  # previous channels block view onto later channels
+        self.include_rand = include_rand
 
     def get_observation_space(self):
-        return spaces.Box(low=0, high=self.num_steps,
-                          shape=(self.num_rays, self.num_chans), dtype=np.uint8)
+        return spaces.Box(low=0, high=1,
+                          shape=(self.num_rays+self.include_rand, self.num_chans), dtype=np.uint8)
 
     def _get_ray_heads(self):
         for i in range(self.num_rays):
@@ -102,35 +111,58 @@ class RayObservable(Observable):
 
     def _check_collision(self, pos, entity_type, entities_l):
         for entity in entities_l:
-            if isinstance(entity, entity_type):
+            if isinstance(entity, entity_type) or (self.env.void_barrier and isinstance(entity, entities.Void) and entity_type == entities.Enemy):
                 if isinstance(entity, entities.Void):
-                    hit = 0 >= pos[0] or pos[0] >= self.env.width or 0 >= pos[1] or pos[0] >= self.env.height
-                    if hit:
-                        print(pos)
-                    return hit
-                else:
-                    if entity.shape != 'circle':
-                        raise Exception('Can only raycast circular entities!')
-                    sq_dist = (pos[0]-entity.pos[0]*self.env.width) ** 2 \
-                        + (pos[1]-entity.pos[1]*self.env.height)**2
-                    if sq_dist < entity.radius**2:
+                    if not self.env.torus_topology and (0 >= pos[0] or pos[0] >= self.env.width or 0 >= pos[1] or pos[1] >= self.env.width):
                         return True
+                else:
+                    if entity.shape == 'circle':
+                        sq_dist = (pos[0]-entity.pos[0]*self.env.width) ** 2 \
+                            + (pos[1]-entity.pos[1]*self.env.height)**2
+                        if sq_dist < entity.radius**2:
+                            return True
+                    elif entity.shape == 'rect':
+                        dot = entities.CircularEntity(self.env)
+                        dot.radius = 1
+                        dot.pos = pos[0]/self.env.width, pos[1]/self.env.height
+                        if sum(dot._get_crash_force_dir(entity)) != 0:
+                            return True
+                    else:
+                        raise Exception(
+                            'Can only raycast circular and rectangular entities!')
         return False
 
+    # Filter out entities, that we sure are out of range
+    # (so we have to do less work for the ray collisions)
     def _get_possible_entities(self):
         entities_l = []
-        if entities.Void in self.chans:
+        if entities.Void in self.chans or self.env.void_barrier:
             entities_l.append(entities.Void(self.env))
         for entity in self.env.entities:
-            sq_dist = ((self.env.agent.pos[0]-entity.pos[0])*self.env.width) ** 2 \
-                + ((self.env.agent.pos[1]-entity.pos[1])*self.env.height) ** 2
-            if sq_dist <= (entity.radius + self.env.agent.radius + self.ray_len)**2:
+            if entity.shape == 'rect':
+                x, y = entity.pos[0]+entity.width/self.env.width / \
+                    2, entity.pos[1]+entity.height/self.env.height/2
+                radius = (entity.width/2 + entity.height/2)*1.0
+            elif entity.shape == 'circle':
+                x, y = entity.pos[0], entity.pos[1]
+                radius = entity.radius
+            else:
+                raise Exception(
+                    'Can only raycast circular and rectangular entities!')
+            sq_dist = ((self.env.agent.pos[0]-x)*self.env.width) ** 2 \
+                + ((self.env.agent.pos[1]-y)*self.env.height) ** 2
+            if sq_dist <= (radius + self.env.agent.getQuasiRadius() + self.ray_len)**2:
                 entities_l.append(entity)  # cannot use yield here!
         return entities_l
 
+    # Ugly, inefficient ray casting
+    # Oh well, it works...
     def get_observation(self):
         entities = self._get_possible_entities()
-        self.rays = np.zeros((self.num_rays, self.num_chans))
+        self.rays = np.zeros((self.num_rays+self.include_rand, self.num_chans))
+        if self.include_rand:
+            for c in range(self.num_chans):
+                self.rays[-1, c] = np.random.rand()
         for r, (hx, hy) in enumerate(self._get_ray_heads()):
             occ_dist = self.num_steps
             for c, entity_type in enumerate(self.chans):
@@ -141,8 +173,10 @@ class RayObservable(Observable):
                     rx, ry = sx + \
                         self.env.agent.pos[0]*self.env.width, sy + \
                         self.env.agent.pos[1]*self.env.height
+                    if self.env.torus_topology:
+                        rx, ry = rx % self.env.width, ry % self.env.height
                     if self._check_collision((rx, ry), entity_type, entities):
-                        self.rays[r, c] = self.num_steps-s
+                        self.rays[r, c] = (self.num_steps-s)/self.num_steps
                         if self.occlusion:
                             occ_dist = s
                         break
@@ -151,19 +185,26 @@ class RayObservable(Observable):
     def draw(self):
         for c, entity_type in enumerate(self.chans):
             for r, (hx, hy) in enumerate(self._get_ray_heads()):
-                s = self.num_steps - self.rays[r, c]
+                s = self.num_steps - self.rays[r, c]*self.num_steps
                 sx, sy = (s+1)*hx/self.num_steps, (s+1)*hy/self.num_steps
                 rx, ry = sx + \
                     self.env.agent.pos[0]*self.env.width, sy + \
                     self.env.agent.pos[1]*self.env.height
+                if self.env.torus_topology:
+                    rx, ry = rx % self.env.width, ry % self.env.height
                 # TODO: How stupid do I want to code?
+                # This instanciates an Object for every Ray-hit,
+                # just to get the color for the visual.
+                # But since this Code will not be executed during training,
+                # I don't think fixing this is an priority...
                 col = entity_type(self.env).col
                 col = int(col[0]/2), int(col[1]/2), int(col[2]/2)
                 pygame.draw.circle(self.env.screen, col, (rx, ry), 3, width=0)
 
 
-def StateObservable(Observable):
-    def __init__(self, coordsAgent=False, speedAgent=False, coordsRelativeToAgent=True, coordsRewards=True, rewardsWhitelist=None, coordsEnemys=True, enemysWhitelist=None, enemysNoBarriers=True, rewardsTimeouts=True):
+class StateObservable(Observable):
+    # Whitelists probably don't work...
+    def __init__(self, coordsAgent=False, speedAgent=False, coordsRelativeToAgent=True, coordsRewards=True, rewardsWhitelist=None, coordsEnemys=True, enemysWhitelist=None, enemysNoBarriers=True, rewardsTimeouts=True, include_rand=True):
         super(StateObservable, self).__init__()
         self._entities = None
         self._timeoutEntities = []
@@ -176,34 +217,41 @@ def StateObservable(Observable):
         self.enemysWhitelist = enemysWhitelist
         self.enemysNoBarriers = enemysNoBarriers
         self.rewardsTimeouts = rewardsTimeouts
+        self.include_rand = include_rand
 
     @property
     def entities(self):
-        if self._entities:
+        if not self._entities == None:
             return self._entities
-        self.rewardsWhitelist = self.rewardsWhitelist or self.env.entities
-        self.enemysWhitelist = self.enemysWhitelist or self.env.entities
+        rewardsWhitelist = self.rewardsWhitelist or self.env.entities
+        enemysWhitelist = self.enemysWhitelist or self.env.entities
         self._entities = []
         if self.coordsAgent:
             self._entities.append(self.env.agent)
         if self.coordRewards:
-            for entity in self.rewardsWhitelist:
+            for entity in rewardsWhitelist:
                 if isinstance(entity, entities.Reward):
                     self._entities.append(entity)
-        if self.coordEnemys:
-            for entity in self.enemysWhitelist:
+        if self.coordsEnemys:
+            for entity in enemysWhitelist:
                 if isinstance(entity, entities.Enemy):
                     if not self.enemysNoBarriers or not isinstance(entity, entities.Barrier):
                         self._entities.append(entity)
-        if self.rewardsTimeout:
-            for entity in self.enemysWhitelist:
+        if self.rewardsTimeouts:
+            for entity in enemysWhitelist:
                 if isinstance(entity, entities.TimeoutReward):
                     self._timeoutEntities.append(entity)
         return self._entities
 
+    def reset(self):
+        self._entities = None
+
     def get_observation_space(self):
-        return spaces.Box(low=0-1*self.coordsRelativeToAgent, high=1,
-                          shape=(len(self.entities)*2+len(self._timeoutEntities) + self.speedAgent), dtype=np.float32)
+        self.reset()
+        num = len(self.entities)*2+len(self._timeoutEntities) + \
+            self.speedAgent*2 + self.include_rand
+        return spaces.Box(low=0-1*(self.coordsRelativeToAgent or self.speedAgent), high=1,
+                          shape=(num,), dtype=np.float64)
 
     def get_observation(self):
         obs = []
@@ -223,10 +271,136 @@ def StateObservable(Observable):
         for entity in self._timeoutEntities:
             obs.append(entity.active)
         if self.speedAgent:
-            obs.append(self.env.speed[0])
-            obs.append(self.env.speed[1])
+            obs.append(self.env.agent.speed[0])
+            obs.append(self.env.agent.speed[1])
+        if self.include_rand:
+            obs.append(np.random.rand())
         self.obs = obs
         return np.array(obs)
 
     def draw(self):
-        pass
+        ofs = (0 + self.env.height/2*self.coordsRelativeToAgent,
+               0 + self.env.width/2*self.coordsRelativeToAgent)
+        if self.coordsRelativeToAgent:
+            pygame.draw.circle(self.env.screen, self.env.agent.col,
+                               (0, self.env.height/2), 3, width=0)
+            pygame.draw.circle(self.env.screen, self.env.agent.col,
+                               (self.env.width/2, 0), 3, width=0)
+        for i in range(int(len(self.obs)/2) - self.speedAgent):
+            x, y = self.obs[i*2], self.obs[i*2+1]
+            col = self.entities[i].col
+            pygame.draw.circle(self.env.screen, col,
+                               (0, y*self.env.height+ofs[0]), 1, width=0)
+            pygame.draw.circle(self.env.screen, col,
+                               (x*self.env.width+ofs[1], 0), 1, width=0)
+
+
+class CompassObservable(Observable):
+    # Usefull for navigation close to an reward.
+    # Works like the StateObservable, but we assign a bigger range of possible input values to those, that are close to zero.
+    # I found that Agents without such an Observable often moved close to a reward and then just jiggled arround, adding a CompassObservable fixes this
+    def __init__(self, coordsRewards=True, rewardsWhitelist=None, coordsEnemys=False, enemysWhitelist=None, enemysNoBarriers=True):
+        super().__init__()
+        self._entities = None
+        self._timeoutEntities = []
+        self.coordRewards = coordsRewards
+        self.rewardsWhitelist = rewardsWhitelist
+        self.coordsEnemys = coordsEnemys
+        self.enemysWhitelist = enemysWhitelist
+        self.enemysNoBarriers = enemysNoBarriers
+
+    @property
+    def entities(self):
+        if not self._entities == None:
+            return self._entities
+        rewardsWhitelist = self.rewardsWhitelist or self.env.entities
+        enemysWhitelist = self.enemysWhitelist or self.env.entities
+        self._entities = []
+        if self.coordRewards:
+            for entity in rewardsWhitelist:
+                if isinstance(entity, entities.Reward):
+                    self._entities.append(entity)
+        if self.coordsEnemys:
+            for entity in enemysWhitelist:
+                if isinstance(entity, entities.Enemy):
+                    if not self.enemysNoBarriers or not isinstance(entity, entities.Barrier):
+                        self._entities.append(entity)
+        return self._entities
+
+    def get_observation_space(self):
+        self.reset()
+        num = len(self.entities)*2
+        return spaces.Box(low=-1, high=1,
+                          shape=(num,), dtype=np.float64)
+
+    def reset(self):
+        self._entities = None
+
+    def get_observation(self):
+        obs = []
+        for entity in self.entities:
+            dx, dy = entity.pos[0] - \
+                self.env.agent.pos[0], entity.pos[1] - self.env.agent.pos[1]
+            l = math.sqrt(dx**2 + dy**2)*2
+            x, y = math.tanh(dx/l), math.tanh(dy/l)
+            obs.append(x)
+            obs.append(y)
+
+        self.obs = obs
+        return np.array(obs)
+
+    def draw(self):
+        ofs = (0 + self.env.height/2,
+               0 + self.env.width/2)
+        if True:
+            pygame.draw.circle(self.env.screen, self.env.agent.col,
+                               (0, self.env.height/2), 3, width=0)
+            pygame.draw.circle(self.env.screen, self.env.agent.col,
+                               (self.env.width/2, 0), 3, width=0)
+        for i in range(int(len(self.obs)/2)):
+            x, y = self.obs[i*2], self.obs[i*2+1]
+            col = self.entities[i].col
+            pygame.draw.circle(self.env.screen, col,
+                               (0, y*self.env.height+ofs[0]), 1, width=0)
+            pygame.draw.circle(self.env.screen, col,
+                               (x*self.env.width+ofs[1], 0), 1, width=0)
+
+
+class CompositionalObservable(Observable):
+    # Used whenever you want to attach multiple Observables to an Env.
+    # We currently flatten the outputs of all attached Observables, so using a CNN though an CompositionalObservable would lead to problems.
+    def __init__(self, observables):
+        super().__init__()
+        self.observables = observables
+
+    def get_observation_space(self):
+        num = 0
+        for i, obs in enumerate(self.observables):
+            space = obs.get_observation_space()
+            num += math.prod(space.shape)
+            if not i:
+                low = space.low.reshape((-1))
+                high = space.high.reshape((-1))
+            else:
+                low = np.hstack((low, space.low.reshape((-1))))
+                high = np.hstack((high, space.high.reshape((-1))))
+        return spaces.Box(low=low, high=high,
+                          shape=(num,), dtype=np.float64)
+
+    def get_observation(self):
+        o = [obs.get_observation().reshape((-1))
+             for obs in self.observables]
+        o = np.hstack(o)
+        return o
+
+    def draw(self):
+        for obs in self.observables:
+            obs.draw()
+
+    def _set_env(self, env):
+        for obs in self.observables:
+            obs._set_env(env)
+
+    def reset(self):
+        for obs in self.observables:
+            obs.reset()

columbus/utils.py

@@ -0,0 +1,42 @@
+from columbus import entities, observables
+
+import random as random_dont_use
+
+
+def parseObs(obsConf):
+    # Parsing Observable Definitions
+    if type(obsConf) == list:
+        obs = []
+        for i, c in enumerate(obsConf):
+            obs.append(parseObs(c))
+        if len(obs) == 1:
+            return obs[0]
+        else:
+            return observables.CompositionalObservable(obs)
+
+    if obsConf['type'] == 'State':
+        conf = {k: v for k, v in obsConf.items() if k not in ['type']}
+        return observables.StateObservable(**conf)
+    elif obsConf['type'] == 'Compass':
+        conf = {k: v for k, v in obsConf.items() if k not in ['type']}
+        return observables.CompassObservable(**conf)
+    elif obsConf['type'] == 'RayCast':
+        chans = []
+        for chan in obsConf.get('chans', []):
+            chans.append(getattr(entities, chan))
+        conf = {k: v for k, v in obsConf.items() if k not in ['type', 'chans']}
+        return observables.RayObservable(chans=chans, **conf)
+    elif obsConf['type'] == 'CNN':
+        conf = {k: v for k, v in obsConf.items() if k not in ['type']}
+        return observables.CnnObservable(**conf)
+    elif obsConf['type'] == 'Dummy':
+        conf = {k: v for k, v in obsConf.items() if k not in ['type']}
+        return observables.Observable(**conf)
+    else:
+        raise Exception('Unknown Observable selected')
+
+
+def soft_int(num):
+    i = int(num)
+    r = num - i
+    return i + int(random_dont_use.random() < r)

configs/Env_A_Local_Optima.yaml

@@ -0,0 +1,78 @@
+name: "DEFAULT"
+
+params:
+  task:
+    task: columbus
+    env_name: ColumbusConfigDefined-v0
+    env_args:
+      observable:
+        - type: State
+          coordsAgent: True
+          speedAgent: True
+          coordsRelativeToAgent: False
+          coordsRewards: True
+          coordsEnemys: False
+          enemysNoBarriers: True
+          rewardsTimeouts: False
+          include_rand: True
+        - type: State
+          coordsAgent: False
+          speedAgent: False
+          coordsRelativeToAgent: True
+          coordsRewards: True
+          coordsEnemys: False
+          enemysNoBarriers: True
+          rewardsTimeouts: False
+          include_rand: True
+        - type: Compass
+        - type: RayCast
+          num_rays: 6
+          chans: [Enemy]
+      entities:
+        - type: RectBarrier
+          damage: 1 #1
+          width: 300
+          height: 120 # 360 - 5%(720)
+          pos: [0, 0]
+        - type: RectBarrier
+          damage: 1 #1
+          width: 300
+          height: 1000
+          pos: [0, 0.25]
+        - type: RectBarrier
+          damage: 1 #1
+          width: 250
+          height: 30
+          pos: [0.55, 0.6]
+        - type: RectBarrier
+          damage: 1 #1
+          width: 30
+          height: 120
+          pos: [0.856, 0.475]
+        - type: RectBarrier
+          num: 0
+          damage: 1 #1
+          width: 50
+          width_rand: 100
+          height: 25
+          height_rand: 100
+        - type: OnceReward
+          reward: 100
+          radius: 20
+          pos: [0.9, 0.8]
+      start_pos: [0.1, 0.21]
+      default_collision_elasticity: 0.8
+      start_score: 10
+      speed_fac: 0.01
+      acc_fac: 0.1
+      die_on_zero: False #True
+      agent_drag: 0.1 # 0.05
+      controll_type: ACC # SPEED
+      aux_reward_max: 1
+      aux_penalty_max: 0.01
+      void_damage: 5 #1
+      terminate_on_reward: True
+      agent_draw_path: True
+      clear_path_on_reset: False
+      max_steps: 450 # 1800
+---

configs/Env_B_Random_Gen.yaml

@@ -0,0 +1,54 @@
+name: "DEFAULT"
+
+params:
+  task:
+    task: columbus
+    num_envs: 8
+    env_args:
+      observable:
+        - type: State
+          coordsAgent: True
+          speedAgent: True
+          coordsRelativeToAgent: False
+          coordsRewards: True
+          coordsEnemys: False
+          enemysNoBarriers: True
+          rewardsTimeouts: False
+          include_rand: True
+        - type: State
+          coordsAgent: False
+          speedAgent: False
+          coordsRelativeToAgent: True
+          coordsRewards: True
+          coordsEnemys: False
+          enemysNoBarriers: True
+          rewardsTimeouts: False
+          include_rand: True
+        - type: Compass
+        - type: RayCast
+          num_rays: 8
+          chans: [Enemy]
+      entities:
+        - type: CircleBarrier
+          num: 8
+          num_rand: 6
+          damage: 20 #20
+          radius: 25
+          radius_rand: 75
+        - type: TeleportingReward
+          num: 1
+          reward: 100 #100
+          radius: 20
+      default_collision_elasticity: 0.8
+      start_score: 50
+      speed_fac: 0.01
+      acc_fac: 0.1
+      die_on_zero: True
+      agent_drag: 0.07 # 0.05
+      controll_type: ACC # SPEED
+      aux_reward_max: 1
+      aux_penalty_max: 0.1
+      void_damage: 5 #1
+      #master_seed: 3.14
+      max_steps: 900 # 30 sec
+---

configs/Env_C_Passage.yaml

@@ -0,0 +1,67 @@
+name: "DEFAULT"
+
+params:
+  task:
+    task: columbus
+    env_name: ColumbusConfigDefined-v0
+    env_args:
+      observable:
+        - type: State
+          coordsAgent: True
+          speedAgent: True
+          coordsRelativeToAgent: False
+          coordsRewards: True
+          coordsEnemys: False
+          enemysNoBarriers: True
+          rewardsTimeouts: False
+          include_rand: True
+        - type: State
+          coordsAgent: False
+          speedAgent: False
+          coordsRelativeToAgent: True
+          coordsRewards: True
+          coordsEnemys: False
+          enemysNoBarriers: True
+          rewardsTimeouts: False
+          include_rand: True
+        - type: RayCast
+          num_rays: 6
+          chans: [Enemy]
+      entities:
+        - type: RectBarrier
+          damage: 10 #1
+          width: 25
+          height: 120 # 360 - 5%(720)
+          pos: [0.45, 0]
+        - type: RectBarrier
+          damage: 10 #1
+          width: 25
+          height: 1000
+          pos: [0.45, 0.25]
+        - type: RectBarrier
+          damage: 10 #1
+          width: 25
+          height: 520 # 360 - 5%(720)
+          pos: [0.55, 0]
+        - type: RectBarrier
+          damage: 10 #1
+          width: 25
+          height: 200
+          pos: [0.55, 0.80]
+        - type: LoopReward
+          num: 1
+          reward: 100 #25
+          radius: 20
+          loop: [[0.125, 0.5, 0.1, 0.5], [0.875, 0.5, 0.1, 0.5]]
+      default_collision_elasticity: 0.8
+      start_score: 10
+      speed_fac: 0.01
+      acc_fac: 0.1
+      die_on_zero: False #True
+      agent_drag: 0.1 # 0.05
+      controll_type: ACC # SPEED
+      aux_reward_max: 1
+      aux_penalty_max: 0.01
+      void_damage: 5 #1
+      agent_draw_path: True
+---

configs/Example_Units.yaml

@@ -0,0 +1,92 @@
+name: "DEFAULT"
+
+# Supported Units:
+# px: Pixels
+# em: 1em = Full Width / Height
+# ct: 100ct = Full Width / Height
+# rx: pixels relative to width
+# ry: pixels relative to height
+# au: 1au = 36px (https://knowyourmeme.com/memes/absolute-unit)
+#
+# When no unit is given, we use the folowing defaults
+# (compatible with legacy behavior)
+# pos: em
+# all other: px
+#
+# ct is the recommendet unit.
+# If you need a unit, that is not responsive in regards to width/height, use au / px.
+
+params:
+  task:
+    task: columbus
+    env_name: ColumbusConfigDefined-v0
+    env_args:
+      observable:
+        - type: State
+          coordsAgent: True
+          speedAgent: True
+          coordsRelativeToAgent: False
+          coordsRewards: True
+          coordsEnemys: False
+          enemysNoBarriers: True
+          rewardsTimeouts: False
+          include_rand: True
+        - type: State
+          coordsAgent: False
+          speedAgent: False
+          coordsRelativeToAgent: True
+          coordsRewards: True
+          coordsEnemys: False
+          enemysNoBarriers: True
+          rewardsTimeouts: False
+          include_rand: True
+        - type: Compass
+        - type: RayCast
+          num_rays: 6
+          chans: [Enemy]
+      entities:
+        - type: RectBarrier
+          num: 1
+          width: 50ct
+          height: 50ct
+          pos: [0ct, 0ct]
+        - type: RectBarrier
+          num: 1
+          width: 50ct
+          height: 50ct
+          pos: [50ct, 50ct]
+        - type: RectBarrier
+          num: 1
+          width: 25rx
+          height: 25ry
+          pos: [0.75em, 30px]
+        - type: RectBarrier
+          num: 1
+          width: 25ry
+          height: 25rx
+          pos: [0.75em, 60px]
+        - type: RectBarrier
+          num: 1
+          width: 20 # defaults to rx (px scaled from x-axis)
+          height: 10 # defaults to ry (px scaled from y-axis)
+          pos: [0.75em, 90px]
+        - type: OnceReward
+          reward: 100
+          radius: 1au
+          pos: [0.3, 0.8] # defaults to em
+      start_pos: [90ct, 20ct]
+      default_collision_elasticity: 0.8
+      start_score: 10
+      speed_fac: 0.01
+      acc_fac: 0.1
+      die_on_zero: False #True
+      agent_drag: 0.1 # 0.05
+      controll_type: ACC # SPEED
+      aux_reward_max: 1
+      aux_penalty_max: 0.01
+      void_damage: 5 #1
+      terminate_on_reward: True
+      agent_draw_path: True
+      clear_path_on_reset: False
+      max_steps: 450 # 1800
+---

configs/Pong.yaml

@@ -0,0 +1,108 @@
+name: "DEFAULT"
+
+params:
+  task:
+    task: columbus
+    env_name: Columbus-v1
+    env_args:
+      observable:
+        - type: State
+          coordsAgent: True
+          speedAgent: True
+          coordsRelativeToAgent: False
+          coordsRewards: True
+          coordsEnemys: False
+          enemysNoBarriers: True
+          rewardsTimeouts: False
+          include_rand: True
+        - type: State
+          coordsAgent: False
+          speedAgent: False
+          coordsRelativeToAgent: True
+          coordsRewards: True
+          coordsEnemys: False
+          enemysNoBarriers: True
+          rewardsTimeouts: False
+          include_rand: True
+        - type: Compass
+        - type: RayCast
+          num_rays: 6
+          chans: [Enemy]
+      entities:
+        - type: Ball
+          radius: 16px
+          pos: [0.8, 0.5]
+          speed: [-0.2, -0.1]
+          speed_rand: [0, 0.2]
+          solid: True
+          collision_elasticity: 3
+          elasticity: 1
+          movable: 1
+          collision_changes_speed: True
+          crash_conservation_of_energy: False
+          min_speed: 0.2
+          max_speed: 0.6
+          draw_path: True
+          draw_path_width: 32
+          draw_path_harm: True
+          drag: 0.00001
+        - type: RectGoal # Good
+          height: 1em
+          width: 10ct
+          pos: [97ct, 0ct]
+          skip_agent_col_check: True
+          col: [0, 255, 0]
+          reward: 30
+          solid: True
+          elasticity: 0.6
+          void_collidable: False
+        - type: Goal # Top
+          radius: 7ct
+          pos: [100ct, 0ct]
+          skip_agent_col_check: True
+          col: [0, 255, 0]
+          reward: 30
+          solid: True
+          elasticity: 0.7
+          void_collidable: False
+        - type: Goal # Bottom
+          radius: 7ct
+          pos: [100ct, 100ct]
+          skip_agent_col_check: True
+          col: [0, 255, 0]
+          reward: 30
+          solid: True
+          elasticity: 0.7
+          void_collidable: False
+        - type: RectGoal # Bad
+          height: 1em
+          width: 3ct
+          pos: [0ct, 0ct]
+          skip_agent_col_check: True
+          col: [255, 0, 0]
+          reward: -45
+          solid: True
+          elasticity: 1000
+          void_collidable: False
+      agent_cls: PongAgent
+      agent_attrs:
+        height: 100
+        width: 30
+        movable: False
+        solid: True
+        elasticity: 0.9
+      exception_for_unsupported_collision: False
+      start_pos: [0.05, 0.5]
+      start_score: 0
+      speed_fac: 0.05
+      acc_fac: 0.1
+      die_on_zero: False #True
+      agent_drag: 0
+      controll_type: SPEED
+      aux_reward_max: 0
+      aux_penalty_max: 0
+      void_damage: 0
+      terminate_on_reward: False
+      agent_draw_path: False
+      clear_path_on_reset: False
+---

configs/example.yaml

@@ -0,0 +1,49 @@
+name: Example
+params:
+ task:
+  env_args:
+      observable:
+        - type: State
+          coordsAgent: True
+          speedAgent: True
+          coordsRelativeToAgent: False
+          coordsRewards: True
+          coordsEnemys: False
+          enemysNoBarriers: True
+          rewardsTimeouts: False
+          include_rand: True
+        - type: State
+          coordsAgent: False
+          speedAgent: False
+          coordsRelativeToAgent: True
+          coordsRewards: True
+          coordsEnemys: False
+          enemysNoBarriers: True
+          rewardsTimeouts: False
+          include_rand: True
+        - type: Compass
+        - type: RayCast
+          num_rays: 8
+          chans: [Enemy]
+      entities:
+        - type: CircleBarrier
+          num: 8
+          num_rand: 6
+          damage: 20
+          radius: 25
+          radius_rand: 75
+        - type: TeleportingReward
+          num: 1
+          reward: 100
+          radius: 20
+      default_collision_elasticity: 0.8
+      start_score: 50
+      speed_fac: 0.01
+      acc_fac: 0.1
+      die_on_zero: True
+      agent_drag: 0.07
+      controll_type: ACC
+      aux_reward_max: 1
+      aux_penalty_max: 0.1
+      void_damage: 5
+---

icon.svg

@@ -0,0 +1,153 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>
+<!-- Created with Inkscape (http://www.inkscape.org/) -->
+
+<svg
+   width="99.153984mm"
+   height="117.12778mm"
+   viewBox="0 0 99.153984 117.12778"
+   version="1.1"
+   id="svg5"
+   xml:space="preserve"
+   inkscape:export-filename="columbus.svg"
+   inkscape:export-xdpi="96"
+   inkscape:export-ydpi="96"
+   sodipodi:docname="columbus.svg"
+   inkscape:version="1.2 (dc2aedaf03, 2022-05-15)"
+   xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
+   xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
+   xmlns="http://www.w3.org/2000/svg"
+   xmlns:svg="http://www.w3.org/2000/svg"><sodipodi:namedview
+     id="namedview7"
+     pagecolor="#505050"
+     bordercolor="#eeeeee"
+     borderopacity="1"
+     inkscape:showpageshadow="0"
+     inkscape:pageopacity="0"
+     inkscape:pagecheckerboard="0"
+     inkscape:deskcolor="#505050"
+     inkscape:document-units="mm"
+     showgrid="false"
+     inkscape:zoom="1.4024329"
+     inkscape:cx="16.043548"
+     inkscape:cy="166.8529"
+     inkscape:window-width="1920"
+     inkscape:window-height="1050"
+     inkscape:window-x="0"
+     inkscape:window-y="0"
+     inkscape:window-maximized="1"
+     inkscape:current-layer="g3514" /><defs
+     id="defs2" /><g
+     inkscape:label="Layer 1"
+     inkscape:groupmode="layer"
+     id="layer1"
+     transform="translate(-50.719907,-147.38589)"><path
+       style="line-height:normal;text-indent:0;text-align:start;text-decoration-line:none;text-decoration-style:solid;text-decoration-color:#000000;text-transform:none;isolation:auto;mix-blend-mode:normal;stroke-width:2.53468"
+       d="m 74.860518,149.35966 c -0.485108,0.0261 -0.952504,0.19102 -1.346548,0.47517 -24.095861,16.78439 -30.037141,49.999 -13.252643,74.09499 9.803749,14.07433 25.215631,21.89775 41.139133,22.67355 v 11.58048 c -10.184721,0.68784 -21.203241,1.40479 -21.203241,1.40479 -2.593164,1.02902 -1.864039,4.88103 0.925813,4.89113 h 22.386418 c 0.27372,0.0452 0.55302,0.0452 0.82674,0 h 22.41116 c 2.78963,-0.0105 3.51837,-3.8622 0.92541,-4.89113 0,0 -11.01854,-0.71695 -21.20326,-1.40479 v -11.57062 c 9.68226,-0.44834 19.35645,-3.48901 27.88649,-9.43075 2.9157,-1.90396 -0.0991,-6.23299 -2.89609,-4.15854 -8.29383,5.77725 -17.75973,8.57401 -27.14389,8.64367 -0.13897,-0.0245 -0.27974,-0.0373 -0.42085,-0.0384 -0.0694,1.5e-4 -0.13874,0.003 -0.20788,0.009 -15.118965,-0.0861 -29.955861,-7.24047 -39.267824,-20.60886 -15.218446,-21.84776 -9.857594,-51.82198 11.990241,-67.04043 2.022521,-1.36404 1.139716,-4.51789 -1.29704,-4.63372 -0.08409,-0.004 -0.168339,-0.004 -0.25243,0 z"
+       id="path219"
+       sodipodi:nodetypes="ccscccccccccccccccscccc" /><path
+       d="m 104.02542,235.41497 v 0 c 23.29851,0 42.1806,-18.88209 42.1806,-42.18061 0,-23.2985 -18.88209,-42.1806 -42.1806,-42.1806 -23.298512,0 -42.180599,18.8821 -42.180599,42.1806 0,23.29852 18.882087,42.18061 42.180599,42.18061 z m 0,3.66787 v 0 c -25.319587,0 -45.848477,-20.52889 -45.848477,-45.84848 0,-25.31958 20.52889,-45.84847 45.848477,-45.84847 9.65374,0 18.61105,2.9843 25.99961,8.0806 11.98999,8.27016 19.84886,22.10203 19.84886,37.76787 0,25.31959 -20.52889,45.84848 -45.84847,45.84848 z m -23.289155,-22.70902 26.893815,-18.95737 1.27696,1.32365 -31.693278,22.43668 -0.06366,0.0617 -1.30746,-1.12193 0.336845,-0.46785 2.158205,-2.82644 -2.365497,3.25476 22.552413,-31.74443 1.226419,1.21459 z m 51.123965,-50.78018 -22.59499,32.86183 -10.451584,-10.46093 z"
+       id="path326"
+       style="stroke-width:0.0935682"
+       sodipodi:nodetypes="ssssssssssssscccccccccccccccc" /><g
+       id="g3514"
+       transform="matrix(0.1421688,0,0,0.1421688,70.795973,160.65622)"
+       style="fill:#000000;fill-opacity:0.2"><g
+         id="g3456"
+         style="fill:#000000;fill-opacity:0.1">
+	<path
+   d="M 151.245,222.446 C 148.054,237.039 135.036,248 119.5,248 c -4.142,0 -7.5,3.357 -7.5,7.5 0,4.143 3.358,7.5 7.5,7.5 23.774,0 43.522,-17.557 46.966,-40.386 14.556,-1.574 27.993,-8.06 38.395,-18.677 2.899,-2.959 2.85,-7.708 -0.109,-10.606 -2.958,-2.897 -7.707,-2.851 -10.606,0.108 C 184.947,202.829 172.643,208 159.5,208 132.757,208 111,186.243 111,159.5 c 0,-4.143 -3.358,-7.5 -7.5,-7.5 -4.142,0 -7.5,3.357 -7.5,7.5 0,32.215 24.119,58.884 55.245,62.946 z"
+   id="path3446"
+   style="fill:#000000;fill-opacity:0.1" />
+	<path
+   d="m 183,287.5 c 0,-4.143 -3.358,-7.5 -7.5,-7.5 -35.014,0 -63.5,28.486 -63.5,63.5 0,0.362 0.013,0.725 0.019,1.088 C 109.23,344.212 106.39,344 103.5,344 c -4.142,0 -7.5,3.357 -7.5,7.5 0,4.143 3.358,7.5 7.5,7.5 26.743,0 48.5,21.757 48.5,48.5 0,4.143 3.358,7.5 7.5,7.5 4.142,0 7.5,-3.357 7.5,-7.5 0,-26.611 -16.462,-49.437 -39.731,-58.867 -0.178,-1.699 -0.269,-3.418 -0.269,-5.133 0,-26.743 21.757,-48.5 48.5,-48.5 4.142,0 7.5,-3.357 7.5,-7.5 z"
+   id="path3448"
+   style="fill:#000000;fill-opacity:0.1" />
+	<path
+   d="m 439,223.5 c 0,-17.075 -6.82,-33.256 -18.875,-45.156 C 422.034,172.236 423,165.918 423,159.5 423,128.626 400.848,102.841 371.606,97.171 373.841,91.6 375,85.628 375,79.5 375,59.943 363.117,43.113 346.194,35.839 317.999,13.383 287.162,0 263.5,0 250.347,0 238.683,6.468 231.5,16.384 224.317,6.468 212.653,0 199.5,0 175.838,0 145.001,13.383 116.806,35.839 99.883,43.113 88,59.943 88,79.5 88,85.628 89.159,91.6 91.394,97.171 62.152,102.841 40,128.626 40,159.5 c 0,6.418 0.965,12.735 2.875,18.844 C 30.82,190.244 24,206.425 24,223.5 c 0,13.348 4.149,25.741 11.213,35.975 C 27.872,270.087 24,282.466 24,295.5 c 0,23.088 12.587,44.242 32.516,55.396 -0.343,2.852 -0.516,5.73 -0.516,8.604 0,31.144 20.315,58.679 49.79,68.063 12.821,21.942 36.175,35.437 61.71,35.437 27.995,0 52.269,-16.181 64,-39.674 11.731,23.493 36.005,39.674 64,39.674 25.535,0 48.889,-13.495 61.71,-35.437 29.475,-9.385 49.79,-36.92 49.79,-68.063 0,-2.874 -0.173,-5.752 -0.516,-8.604 C 426.413,339.742 439,318.588 439,295.5 439,282.466 435.128,270.087 427.787,259.475 434.851,249.241 439,236.848 439,223.5 Z M 167.5,448 c -21.029,0 -40.191,-11.594 -50.009,-30.256 -0.973,-1.849 -2.671,-3.208 -4.688,-3.751 C 88.19,407.369 71,384.961 71,359.5 c 0,-3.81 0.384,-7.626 1.141,-11.344 0.702,-3.447 -1.087,-6.92 -4.302,-8.35 C 50.32,332.018 39,314.626 39,295.5 39,286.801 41.256,278.486 45.561,271.121 56.757,280.992 71.436,287 87.5,287 91.642,287 95,283.643 95,279.5 95,275.357 91.642,272 87.5,272 60.757,272 39,250.243 39,223.5 c 0,-14.396 6.352,-27.964 17.428,-37.221 2.5,-2.09 3.365,-5.555 2.14,-8.574 C 56.2,171.869 55,165.744 55,159.5 55,132.757 76.757,111 103.5,111 c 26.743,0 48.5,21.757 48.5,48.5 0,4.143 3.358,7.5 7.5,7.5 4.142,0 7.5,-3.357 7.5,-7.5 C 167,125.858 140.698,98.257 107.579,96.145 104.577,91.127 103,85.421 103,79.5 c 0,-13.369 8.116,-24.875 19.678,-29.859 0.447,-0.133 0.885,-0.307 1.308,-0.527 C 127.568,47.752 131.447,47 135.5,47 c 12.557,0 23.767,7.021 29.256,18.325 1.81,3.727 6.298,5.281 10.023,3.47 3.726,-1.809 5.28,-6.296 3.47,-10.022 C 171.983,45.87 160.124,36.596 146.467,33.311 165.609,21.631 184.454,15 199.5,15 213.009,15 224,25.99 224,39.5 v 97.051 C 217.261,131.205 208.75,128 199.5,128 c -4.142,0 -7.5,3.357 -7.5,7.5 0,4.143 3.358,7.5 7.5,7.5 13.509,0 24.5,10.99 24.5,24.5 v 180.279 c -9.325,-12.031 -22.471,-21.111 -37.935,-25.266 -3.999,-1.071 -8.114,1.297 -9.189,5.297 -1.075,4.001 1.297,8.115 5.297,9.189 C 206.8,343.616 224,366.027 224,391.5 224,422.654 198.654,448 167.5,448 Z M 395.161,339.807 c -3.215,1.43 -5.004,4.902 -4.302,8.35 0.757,3.718 1.141,7.534 1.141,11.344 0,25.461 -17.19,47.869 -41.803,54.493 -2.017,0.543 -3.716,1.902 -4.688,3.751 C 335.691,436.406 316.529,448 295.5,448 264.346,448 239,422.654 239,391.5 c 0,-2.109 -0.098,-4.2 -0.281,-6.271 0.178,-0.641 0.281,-1.314 0.281,-2.012 V 135.5 c 0,-13.51 10.991,-24.5 24.5,-24.5 4.142,0 7.5,-3.357 7.5,-7.5 0,-4.143 -3.358,-7.5 -7.5,-7.5 -9.25,0 -17.761,3.205 -24.5,8.551 V 39.5 C 239,25.99 249.991,15 263.5,15 c 15.046,0 33.891,6.631 53.033,18.311 -13.657,3.284 -25.516,12.559 -31.782,25.462 -1.81,3.727 -0.256,8.214 3.47,10.022 3.726,1.81 8.213,0.257 10.023,-3.47 C 303.733,54.021 314.943,47 327.5,47 c 4.053,0 7.933,0.752 11.514,2.114 0.422,0.22 0.86,0.393 1.305,0.526 C 351.883,54.624 360,66.13 360,79.5 c 0,5.921 -1.577,11.627 -4.579,16.645 C 322.302,98.257 296,125.858 296,159.5 c 0,4.143 3.358,7.5 7.5,7.5 4.142,0 7.5,-3.357 7.5,-7.5 0,-26.743 21.757,-48.5 48.5,-48.5 26.743,0 48.5,21.757 48.5,48.5 0,6.244 -1.2,12.369 -3.567,18.205 -1.225,3.02 -0.36,6.484 2.14,8.574 11.075,9.257 17.427,22.825 17.427,37.221 0,26.743 -21.757,48.5 -48.5,48.5 -4.142,0 -7.5,3.357 -7.5,7.5 0,4.143 3.358,7.5 7.5,7.5 16.064,0 30.743,-6.008 41.939,-15.879 4.306,7.365 6.561,15.68 6.561,24.379 0,19.126 -11.32,36.518 -28.839,44.307 z"
+   id="path3450"
+   style="fill:#000000;fill-opacity:0.1" />
+	<path
+   d="m 355.7779,243.7221 c -15.536,0 -24.8319,-14.6831 -28.0229,-29.2761 C 358.881,210.384 383,183.715 383,151.5 c 0,-4.143 -3.358,-7.5 -7.5,-7.5 -4.142,0 -7.5,3.357 -7.5,7.5 0,26.743 -21.757,48.5 -48.5,48.5 -13.143,0 -25.447,-5.171 -34.646,-14.561 -2.898,-2.958 -7.647,-3.007 -10.606,-0.108 -2.959,2.899 -3.008,7.647 -0.109,10.606 10.402,10.617 23.839,17.103 38.395,18.677 3.444,22.829 19.4699,44.1081 43.2439,44.1081 4.142,0 7.5,-3.357 7.5,-7.5 0,-4.143 -3.358,-7.5 -7.5,-7.5 z"
+   id="path3452"
+   style="fill:#000000;fill-opacity:0.1"
+   sodipodi:nodetypes="scsssscsccsss" />
+	<path
+   d="m 335.5,328 c -2.89,0 -5.73,0.212 -8.519,0.588 0.006,-0.363 0.019,-0.726 0.019,-1.088 0,-35.014 -28.486,-63.5 -63.5,-63.5 -4.142,0 -7.5,3.357 -7.5,7.5 0,4.143 3.358,7.5 7.5,7.5 26.743,0 48.5,21.757 48.5,48.5 0,1.714 -0.091,3.434 -0.269,5.133 C 288.462,342.063 272,364.889 272,391.5 c 0,4.143 3.358,7.5 7.5,7.5 4.142,0 7.5,-3.357 7.5,-7.5 0,-26.743 21.757,-48.5 48.5,-48.5 4.142,0 7.5,-3.357 7.5,-7.5 0,-4.143 -3.358,-7.5 -7.5,-7.5 z"
+   id="path3454"
+   style="fill:#000000;fill-opacity:0.1" />
+</g><g
+         id="g3458"
+         style="fill:#000000;fill-opacity:0.2">
+</g><g
+         id="g3460"
+         style="fill:#000000;fill-opacity:0.2">
+</g><g
+         id="g3462"
+         style="fill:#000000;fill-opacity:0.2">
+</g><g
+         id="g3464"
+         style="fill:#000000;fill-opacity:0.2">
+</g><g
+         id="g3466"
+         style="fill:#000000;fill-opacity:0.2">
+</g><g
+         id="g3468"
+         style="fill:#000000;fill-opacity:0.2">
+</g><g
+         id="g3470"
+         style="fill:#000000;fill-opacity:0.2">
+</g><g
+         id="g3472"
+         style="fill:#000000;fill-opacity:0.2">
+</g><g
+         id="g3474"
+         style="fill:#000000;fill-opacity:0.2">
+</g><g
+         id="g3476"
+         style="fill:#000000;fill-opacity:0.2">
+</g><g
+         id="g3478"
+         style="fill:#000000;fill-opacity:0.2">
+</g><g
+         id="g3480"
+         style="fill:#000000;fill-opacity:0.2">
+</g><g
+         id="g3482"
+         style="fill:#000000;fill-opacity:0.2">
+</g><g
+         id="g3484"
+         style="fill:#000000;fill-opacity:0.2">
+</g><g
+         id="g3486"
+         style="fill:#000000;fill-opacity:0.2">
+</g></g></g><style
+     type="text/css"
+     id="style139">
+	.st0{fill:#BB9645;}
+	.st1{fill:#0A0B0B;}
+	.st2{fill:#FFFFFF;}
+	.st3{fill:#231F20;}
+	.st4{fill:url(#SVGID_1_);}
+	.st5{fill:url(#SVGID_2_);}
+	.st6{fill:url(#SVGID_3_);}
+	.st7{fill:url(#SVGID_4_);}
+	.st8{fill:url(#SVGID_5_);}
+	.st9{fill:url(#SVGID_6_);}
+	.st10{fill:url(#SVGID_7_);}
+	.st11{fill:url(#SVGID_8_);}
+	.st12{fill:url(#SVGID_9_);}
+	.st13{fill:url(#SVGID_10_);}
+	.st14{fill:url(#SVGID_11_);}
+	.st15{fill:#1B1B1B;}
+	.st16{fill:url(#SVGID_12_);}
+	.st17{fill:url(#SVGID_13_);}
+	.st18{fill:url(#SVGID_14_);}
+	.st19{fill:url(#SVGID_15_);}
+	.st20{fill:url(#SVGID_16_);}
+	.st21{fill:url(#SVGID_17_);}
+	.st22{fill:url(#SVGID_18_);}
+	.st23{fill:url(#SVGID_19_);}
+	.st24{fill:url(#SVGID_20_);}
+	.st25{fill:url(#SVGID_21_);}
+	.st26{fill:url(#SVGID_22_);}
+</style></svg>

requirements.txt

@@ -0,0 +1,2 @@
+gym
+pygame