Made path_decay configurable

rendering

.gitignore

@@ -1,4 +1,5 @@
 __pychache__
 *.pyc
 *.pyo
-*.egg-info
+*.egg-info
+.venv

LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2022 Dominik Roth
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

README.md

@@ -1,19 +1,57 @@
 # Project Columbus
 
-![Icon](./icon.svg)
+<p align='center'>
+  <img src='./icon.svg'>
+</p>
 
-Project Columbus is a framework for trivial 2D OpenAI Gym environments that are supposed to test a agents ability to solve tasks that require different forms of exploration effectively and efficiently.  
+Project Columbus is a framework for trivial 2D OpenAI Gym environments that are supposed to test a agents ability to solve tasks that require different forms of exploration effectively and efficiently.
 
-![Screenshot](./img_README.png)
+## Installation
+
+(If you want to install Columbus as a dependency for metastable-baselines, activate (source) the venv from metastable-baselines before running this command.)
+
+```
+pip install -e .
+```
+
+## Overview
+
+Columbus.pdf contains a overview of columbus.
+
+## Layout of the Repo
 
 ### env.py
-Contains the ColumbusEnv. New envs are implemented by subclassing ColumbusEnv and expanding _init_ and overriding _setup_.
+
+![Screenshot](./img_README.png)  
+Contains the ColumbusEnv.
+There exist two ways to implement new envs:
+
+- Subclassing ColumbusEnv and expanding _init_ and overriding _setup_.
+- Using the ColumbusConfigDefined with a desired configuration. This makes configuring ColumbusEnvs via ClusterWorks2-configs possible. (See configs/example.yaml for an example of how the parameters are supposed to look like (uses yaml format)
+- We now support using units (px, em, ct) in config files, examples can be found in configs/Example_Units.yaml
+- The environments used in my thesis can also be found in configs/
+
+##### Some caveats / infos
+
+- If you want to render to a window (pygame-gui) call render with mode='human'
+- If you want visualize the covariance you have to supply the cholesky-decomp of the cov-matrix to render
+- If you want to render into a mp4, you have to call render with a mode!='human' and assemble/encode the returned frames yourself into a mp4/webm/...
+- Even while the agent plays, some keyboard-inputs are possible (to test the agents reaction to situations he would never enter by itself. Look at \_handle_user_input in env.py for avaible keys)
+- The sampling-rate of the physics engine is bound to the frame-rate of the rendering engine (1:1). This means too low fps / too fast agents / too thin barriers will lead to the agent tunneling through barriers. You can fix this by setting a higher agent-drag (which decreases the maximum speed) or making barriers thicker. A feature allowing the physics engine to sample multiple smaller steps within a single rendering step could be added in the future.
 
 ### entities.py
+
 Contains all implemented entities (e.g. the Agent, Rewards and Enemies)
 
 ### observables.py
+
 Contains all 'oberservables'. These are attached to envs to define what kind of output is given to the agent. This way environments can be designed independently from the observation machanism that is used by the agent to play it.
 
+##### Some caveats
+
+- CNNObservable seems to be broken currently. (Fixing it is also no priority for me)
+- RayObservable is using a naive ray-marching (basicaly just line-sweeping). For large amounts of rays this turn out to be the computational bottleneck of the environment. Switching to a more efficient algorithm (based on euclidean formulars and line intersects) would be possible in the future...
+
 ### humanPlayer.py
-Allows environments to be played by a human using mouse input.
+
+Allows environments to be played by a human using mouse input. Now even works for ColumbusConfigDefined.

columbus/entities.py

@@ -3,28 +3,61 @@ import math
 
 
 class Entity(object):
+    def __call__(cls, *args, **kwargs):
+        obj = type.__call__(cls, *args, **kwargs)
+        obj.__post_init__()
+        return obj
+
     def __init__(self, env):
+        self.shape = None
         self.env = env
         self.pos = (env.random(), env.random())
+        self.last_pos = None
         self.speed = (0, 0)
         self.acc = (0, 0)
         self.drag = 0
-        self.radius = 10
         self.col = (255, 255, 255)
-        self.shape = 'circle'
+        self.solid = False
+        self.movable = False  # False = Non movable, True = Movable, x>1: lighter movable
+        self.void_collidable = False
+        self.elasticity = 1
+        self.collision_changes_speed = self.env.controll_type == 'ACC'
+        self.collision_elasticity = self.env.default_collision_elasticity
+        self._crash_list = []
+        self._coll_add_pushback = 0
+        self.crash_conservation_of_energy = True
+        self.draw_path = False
+        self.draw_path_col = [int(c/5) for c in self.col]
+        self.draw_path_width = 2
+        self.draw_path_harm = False
+        self.draw_path_harm_col = [c for c in self.draw_path_col]
+        self.draw_path_harm_col[0] += int(255/3)
+        self.min_speed = 0
+        self.max_speed = math.inf
+
+    def __post_init__(self):
+        pass
 
     def physics_step(self):
         x, y = self.pos
         vx, vy = self.speed
         ax, ay = self.acc
         vx, vy = vx+ax*self.env.acc_fac,  vy+ay*self.env.acc_fac
+        speeds = math.sqrt(vx**2 + vy**2)
+        if speeds < self.min_speed:
+            vx, vy = vx/speeds*self.min_speed, vy/speeds*self.min_speed
+        if speeds > self.max_speed:
+            vx, vy = vx/speeds*self.max_speed, vy/speeds*self.max_speed
         x, y = x+vx*self.env.speed_fac, y+vy*self.env.speed_fac
-        if x > 1 or x < 0:
+        if not self.env.torus_topology and self.void_collidable:
-            x = min(max(x, 0), 1)
+            if x > 1 or x < 0:
-            vx = 0
+                x, y, vx, vy = self.calc_void_collision(x < 0, x, y, vx, vy)
-        if y > 1 or y < 0:
+            if y > 1 or y < 0:
-            y = min(max(y, 0), 1)
+                x, y, vx, vy = self.calc_void_collision(
-            vy = 0
+                    2 + (x < 0), x, y, vx, vy)
+        else:
+            x = x % 1
+            y = y % 1
         self.speed = vx/(1+self.drag), vy/(1+self.drag)
         self.pos = x, y
 
@@ -34,26 +67,281 @@ class Entity(object):
     def step(self):
         self.controll_step()
         self.physics_step()
+        self._crash_list = []
 
     def draw(self):
-        x, y = self.pos
+        self._draw_path()
-        pygame.draw.circle(self.env.surface, self.col,
-                           (x*self.env.width, y*self.env.height), self.radius, width=0)
 
-    def on_collision(self, other):
+    def _draw_path(self):
+        if self.draw_path and self.last_pos:
+            col = self.draw_path_col
+            if self.draw_path_harm:
+                if self.env.gotHarm:
+                    col = self.draw_path_harm_col
+            pygame.draw.line(self.env.path_overlay, col,
+                             (self.last_pos[0]*self.env.width, self.last_pos[1]*self.env.height), (self.pos[0]*self.env.width, self.pos[1]*self.env.height), self.draw_path_width)
+            pygame.draw.circle(self.env.path_overlay, col,
+                               (self.pos[0]*self.env.width, self.pos[1]*self.env.height), max(0, self.draw_path_width/2-3))
+        self.last_pos = self.pos[0], self.pos[1]
+
+    def on_collision(self, other, depth):
+        if self.solid and other.solid:
+            if self.movable:
+                self.on_crash(other, depth)
+
+    def on_crash(self, other, depth):
+        if other in self._crash_list:
+            return
+        self._crash_list.append(other)
+        force_dir = self._get_crash_force_dir(other)
+        #print(force_dir, depth)
+        force_dir_len = math.sqrt(force_dir[0]**2+force_dir[1]**2)
+        if force_dir_len == 0:
+            return
+        force_dir = force_dir[0]/force_dir_len, force_dir[1]/force_dir_len
+        if not self.env.torus_topology:
+            if self == self.env.agent:
+                agent = self
+            elif other == self.env.agent:
+                agent = other
+            else:
+                agent = None
+            if agent:
+                if agent.pos[0] > 0.99 or agent.pos[0] < 0.01:
+                    force_dir = force_dir[0], force_dir[1] * 2
+                if agent.pos[1] > 0.99 or agent.pos[1] < 0.01:
+                    force_dir = force_dir[0] * 2, force_dir[1]
+        depth *= 1.0*self.movable/(self.movable + other.movable)/2
+        depth /= other.elasticity
+        force_vec = force_dir[0]*depth/self.env.width, \
+            force_dir[1]*depth/self.env.height
+        if sum([abs(f) for f in force_vec]) > 0.005:
+            self.pos = self.pos[0] + force_vec[0], self.pos[1] + force_vec[1]
+            if self._coll_add_pushback:
+                self.pos = self.pos[0] - self.env.inp[0]*self._coll_add_pushback * \
+                    self.env.speed_fac, self.pos[1] - self.env.inp[1] * \
+                    self._coll_add_pushback*self.env.speed_fac
+        if self.collision_changes_speed:
+            oldspeed = math.sqrt(self.speed[0]**2+self.speed[1]**2)
+            self.speed = self.speed[0] + \
+                force_vec[0]*self.collision_elasticity/self.env.speed_fac, self.speed[1] + \
+                force_vec[1]*self.collision_elasticity/self.env.speed_fac
+            newspeed = math.sqrt(self.speed[0]**2+self.speed[1]**2)
+            if self.crash_conservation_of_energy and newspeed > oldspeed*1.1:
+                self.speed = self.speed[0]/newspeed*1.1 * \
+                    oldspeed, self.speed[1]/newspeed*oldspeed*1.1
+
+    def _get_crash_force_dir(self, other):
+        if 1 == 1:  # linter hack
+            raise Exception(
+                '[!] No collision-logic implemented for shape"'+str(self.shape)+'"')
+
+    def on_collect(self, other):
         pass
 
+    def on_collected(self):
+        pass
+
+    def calc_void_collision(self, dir, x, y, vx, vy):
+        if dir < 2:
+            x = min(max(x, 0), 1)
+            vx = -vx*self.collision_elasticity*0.5*self.collision_changes_speed
+        else:
+            y = min(max(y, 0), 1)
+            vy = -vy*self.collision_elasticity*0.5*self.collision_changes_speed
+        return x, y, vx, vy
+
     def kill(self):
         self.env.kill_entity(self)
 
+    def getQuasiRadius(self):
+        raise Exception()
 
-class Agent(Entity):
+    def getTop(self):
+        raise Exception()
+
+    def getBottom(self):
+        raise Exception()
+
+    def getLeft(self):
+        raise Exception()
+
+    def getRight(self):
+        raise Exception()
+
+    def getCenter(self):
+        raise Exception()
+
+
+class CircularEntity(Entity):
+    def __init__(self, env):
+        super().__init__(env)
+        self.shape = 'circle'
+        self.radius = 10
+
+    def draw(self):
+        super().draw()
+        x, y = self.pos
+        pygame.draw.circle(self.env.surface, self.col,
+                           (x*self.env.width, y*self.env.height), self.radius, width=0)
+
+    def _get_crash_force_dir(self, other):
+        if other.shape == 'circle':
+            return self.pos[0] - other.pos[0], self.pos[1] - other.pos[1]
+        elif other.shape == 'rect':
+            pad = 0
+            edge_size = min(self.radius, min(
+                other.width/3, other.height/3)) + 1
+
+            x, y = self.pos
+            x, y = x*self.env.height, y*self.env.width
+            left, top = x - self.radius + pad, y - self.radius + pad
+            right, bottom = x + self.radius - pad, y + self.radius - pad
+            lrcenter, tbcenter = x, y
+
+            ox, oy = other.pos
+            ox, oy = ox*self.env.height, oy*self.env.width
+            oleft, otop = ox + pad, oy + pad
+            oright, obottom = ox + other.width - pad, oy + other.height - pad
+            olrcenter, otbcenter = ox + other.width/2, oy + other.height/2
+
+            lr, tb = 0, 0
+
+            if otop < bottom and obottom > bottom:
+                # col from top
+                tb = otop - bottom
+                #print('t', tb)
+            elif top < obottom and top > otop:
+                # col from bottom
+                tb = - top + obottom
+                #print('b', tb)
+
+            if right > oleft and right < oright:
+                # col from left
+                lr = oleft - right
+                #print('l', lr)
+            elif left < oright and left > oleft:
+                # col from right
+                lr = - left + oright
+                #print('r', lr)
+
+            if lr != 0 and tb != 0:
+                if abs(abs(tb) - abs(lr)) < edge_size:
+                    if abs(tb) < abs(lr):
+                        return lr/5, tb
+                    else:
+                        return lr, tb/5
+                if abs(tb) < abs(lr):
+                    return 0, tb
+                else:
+                    return lr, 0
+
+            return 0, 0
+        else:
+            raise Exception(
+                '[!] Shape "circle" does not know how to collide with shape "'+str(other.shape)+'"')
+
+    def getQuasiRadius(self):
+        return self.radius
+
+    def getTop(self):
+        return self.pos[1]*self.env.height - self.radius
+
+    def getBottom(self):
+        return self.pos[1]*self.env.height + self.radius
+
+    def getLeft(self):
+        return self.pos[0]*self.env.width - self.radius
+
+    def getRight(self):
+        return self.pos[0]*self.env.width + self.radius
+
+    def getCenter(self):
+        return self.pos[0]*self.env.width, self.pos[1]*self.env.height
+
+
+class RectangularEntity(Entity):
+    def __init__(self, env):
+        super().__init__(env)
+        self.shape = 'rect'
+        self.width = 10
+        self.height = 10
+
+    def draw(self):
+        super().draw()
+        x, y = self.pos
+        rect = pygame.Rect(x*self.env.width, y *
+                           self.env.width, self.width, self.height)
+        pygame.draw.rect(self.env.surface, self.col,
+                         rect, width=0)
+
+    def _get_crash_force_dir(self, other):
+        raise Exception(
+            '[!] Collisions in this direction not implemented for shape "rectangle"')
+
+    def physics_step(self):
+        x, y = self.pos
+        vx, vy = self.speed
+        ax, ay = self.acc
+        vx, vy = vx+ax*self.env.acc_fac,  vy+ay*self.env.acc_fac
+        speeds = math.sqrt(vx**2 + vy**2)
+        if speeds < self.min_speed:
+            vx, vy = vx/speeds*self.min_speed, vy/speeds*self.min_speed
+        if speeds > self.max_speed:
+            vx, vy = vx/speeds*self.max_speed, vy/speeds*self.max_speed
+        x, y = x+vx*self.env.speed_fac, y+vy*self.env.speed_fac
+        if not self.env.torus_topology and self.void_collidable:
+            if x+(self.width/self.env.width) > 1 or x < 0:
+                if x < 0:
+                    x, y, vx, vy = self.calc_void_collision(
+                        x < 0, x, y, vx, vy)
+                else:
+                    x, y, vx, vy = self.calc_void_collision(
+                        x < 0, x+(self.width/self.env.width), y, vx, vy)
+                    x -= (self.width/self.env.width)
+            if y+(self.height/self.env.height) > 1 or y < 0:
+                if y < 0:
+                    x, y, vx, vy = self.calc_void_collision(
+                        2 + (x < 0), x, y, vx, vy)
+                else:
+                    x, y, vx, vy = self.calc_void_collision(
+                        2 + (x < 0), x, y+(self.height/self.env.height), vx, vy)
+                    y -= (self.height/self.env.height)
+        else:
+            x = x % 1
+            y = y % 1
+        self.speed = vx/(1+self.drag), vy/(1+self.drag)
+        self.pos = x, y
+
+    def getQuasiRadius(self):
+        return self.width + self.height
+
+    def getTop(self):
+        return self.pos[1]*self.env.height
+
+    def getBottom(self):
+        return self.pos[1]*self.env.height + self.height
+
+    def getLeft(self):
+        return self.pos[0]*self.env.width
+
+    def getRight(self):
+        return self.pos[0]*self.env.width*self.env.height + self.width
+
+    def getCenter(self):
+        return self.pos[0]*self.env.width+self.width/2, self.pos[1]*self.env.height+self.height/2
+
+
+class Agent(CircularEntity):
     def __init__(self, env):
         super(Agent, self).__init__(env)
         self.pos = (0.5, 0.5)
         self.col = (0, 0, 255)
         self.drag = self.env.agent_drag
         self.controll_type = self.env.controll_type
+        self.solid = True
+        self.movable = True
+        self.void_collidable = True
 
     def controll_step(self):
         self._read_input()
@@ -68,13 +356,39 @@ class Agent(Entity):
             raise Exception('Unsupported controll_type')
 
 
+# Does not work! Don't use!
+class PongAgent(RectangularEntity):
+    def __init__(self, env):
+        super(PongAgent, self).__init__(env)
+        self.pos = (0.5, 0.5)
+        self.col = (0, 0, 255)
+        self.drag = self.env.agent_drag
+        self.controll_type = self.env.controll_type
+        self.solid = True
+        self.movable = True
+
+    def controll_step(self):
+        self._read_input()
+        self.env.check_collisions_for(self)
+
+    def _read_input(self):
+        if self.controll_type == 'SPEED':
+            self.speed = 0, self.env.inp[1] - 0.5
+        elif self.controll_type == 'ACC':
+            self.acc = 0, self.env.inp[1] - 0.5
+        else:
+            raise Exception('Unsupported controll_type')
+
+
 class Enemy(Entity):
     def __init__(self, env):
         super(Enemy, self).__init__(env)
         self.col = (255, 0, 0)
         self.damage = 100
+        self.radiateDamage = True
 
-    def on_collision(self, other):
+    def on_collision(self, other, depth):
+        super().on_collision(other, depth)
         if isinstance(other, Agent):
             self.env.new_reward -= self.damage
 
@@ -82,14 +396,21 @@ class Enemy(Entity):
 class Barrier(Enemy):
     def __init__(self, env):
         super(Barrier, self).__init__(env)
+        self.solid = True
+        self.movable = False
 
 
-class CircleBarrier(Barrier):
+class CircleBarrier(Barrier, CircularEntity):
     def __init__(self, env):
         super(CircleBarrier, self).__init__(env)
 
 
-class Chaser(Enemy):
+class RectBarrier(Barrier, RectangularEntity):
+    def __init__(self, env):
+        super().__init__(env)
+
+
+class Chaser(Enemy, CircularEntity):
     def __init__(self, env):
         super(Chaser, self).__init__(env)
         self.target = self.env.agent
@@ -129,21 +450,25 @@ class FlyingChaser(Chaser):
         self.acc = arrow[0] * self.chase_acc, arrow[1] * self.chase_acc
 
 
-class Reward(Entity):
+class Collectable(CircularEntity):
     def __init__(self, env):
-        super(Reward, self).__init__(env)
+        super(Collectable, self).__init__(env)
-        self.col = (0, 255, 0)
         self.avaible = True
         self.enforce_not_on_barrier = False
         self.reward = 10
+        self.collectors = []
 
-    def on_collision(self, other):
+    def on_collision(self, other, depth):
-        if isinstance(other, Agent):
+        super().on_collision(other, depth)
-            self.on_collect()
+        if isinstance(other, Barrier):
-        elif isinstance(other, Barrier):
             self.on_barrier_collision()
+        else:
+            for Col in self.collectors:
+                if isinstance(other, Col):
+                    other.on_collect(self)
+                    self.on_collected()
 
-    def on_collect(self):
+    def on_collected(self):
         self.env.new_reward += self.reward
 
     def on_barrier_collision(self):
@@ -152,12 +477,50 @@ class Reward(Entity):
             self.env.check_collisions_for(self)
 
 
+class RectCollectable(RectangularEntity):
+    def __init__(self, env):
+        super(RectCollectable, self).__init__(env)
+        self.avaible = True
+        self.enforce_not_on_barrier = False
+        self.reward = 10
+        self.collectors = []
+
+    def on_collision(self, other, depth):
+        super().on_collision(other, depth)
+        if isinstance(other, Barrier):
+            self.on_barrier_collision()
+        else:
+            for Col in self.collectors:
+                if isinstance(other, Col):
+                    other.on_collect(self)
+                    self.on_collected()
+
+    def on_collected(self):
+        self.env.new_reward += self.reward
+
+    def on_barrier_collision(self):
+        if self.enforce_not_on_barrier:
+            self.pos = (self.env.random(), self.env.random())
+            self.env.check_collisions_for(self)
+
+
+class Reward(Collectable):
+    def __init__(self, env):
+        super(Reward, self).__init__(env)
+        self.col = (0, 255, 0)
+        self.reward = 10
+        self.collectors = [Agent]
+
+
 class OnceReward(Reward):
     def __init__(self, env):
         super(OnceReward, self).__init__(env)
         self.reward = 500
 
-    def on_collect(self):
+    def on_collected(self):
+        # Force rerender of value func (even in static envs)
+        self.env._invalidate_value_map()
+
         self.env.new_abs_reward += self.reward
         self.kill()
 
@@ -168,12 +531,51 @@ class TeleportingReward(OnceReward):
         self.enforce_not_on_barrier = True
         self.env.check_collisions_for(self)
 
-    def on_collect(self):
+    def on_collected(self):
+        # Force rerender of value func (even in static envs)
+        self.env._invalidate_value_map()
+
         self.env.new_abs_reward += self.reward
         self.pos = (self.env.random(), self.env.random())
         self.env.check_collisions_for(self)
 
 
+class LoopReward(OnceReward):
+    def __init__(self, env):
+        super().__init__(env)
+        self.loop = [[0.25, 0.5], [0.75, 0.5]]
+        self.state = 0
+        self.jump_to_state()
+        self.barrier_physics = False
+
+    def jump_to_state(self):
+        # Force rerender of value func (even in static envs)
+        self.env._invalidate_value_map()
+
+        pos_vec = [v for v in self.loop[self.state]]
+        if len(pos_vec) == 4:
+            pos_vec = pos_vec[0] + pos_vec[2] * \
+                (self.env.random()-0.5), pos_vec[1] + \
+                pos_vec[3]*(self.env.random()-0.5)
+        self.pos = pos_vec
+
+    def next_state(self):
+        self.state = (self.state + 1) % len(self.loop)
+
+    def jump_next(self):
+        self.next_state()
+        self.jump_to_state()
+
+    def on_collected(self):
+        self.env.new_abs_reward += self.reward
+        self.jump_next()
+
+    def physics_step(self):
+        if self.barrier_physics:
+            self.env.check_collisions_for(self)
+        super().physics_step()
+
+
 class TimeoutReward(OnceReward):
     def __init__(self, env):
         super(TimeoutReward, self).__init__(env)
@@ -188,14 +590,96 @@ class TimeoutReward(OnceReward):
         else:
             self.col = (50, 100, 50)
 
-    def on_collect(self):
+    def on_collected(self):
         if self.avaible:
+            # Force rerender of value func (even in static envs)
+            self.env._invalidate_value_map()
+
             self.env.new_abs_reward += self.reward
             self.set_avaible(False)
             self.env.timers.append((self.timeout, self.set_avaible, True))
 
 
+class Ball(CircularEntity):
+    def __init__(self, env):
+        super(Ball, self).__init__(env)
+        self.col = (255, 128, 0)
+        self.drag = 0.0025
+        self.solid = True
+        self.movable = 10
+        self.elasticity = 1
+        self.collision_changes_speed = True
+        self.wall_reflect_damping = 0.1
+
+    def calc_void_collision(self, dir, x, y, vx, vy):
+        if dir < 2:
+            x = min(max(x, 0), 1)
+            vx = -(vx/(1+self.wall_reflect_damping))
+        else:
+            y = min(max(y, 0), 1)
+            vy = -(vy/(1+self.wall_reflect_damping))
+        return x, y, vx, vy
+
+    def physics_step(self):
+        self.env.check_collisions_for(self)
+        super().physics_step()
+
+
+class Goal(Collectable):
+    def __init__(self, env):
+        super(Goal, self).__init__(env)
+        self.col = (0, 200, 0)
+        self.reward = 500
+        self.radius = 20
+        self.collectors = [Ball]
+
+
+class RectGoal(RectCollectable):
+    def __init__(self, env):
+        super(RectGoal, self).__init__(env)
+        self.col = (0, 200, 0)
+        self.reward = 500
+        self.collectors = [Ball]
+
+
+class TeleportingGoal(Goal):
+    def __init__(self, env):
+        super(TeleportingGoal, self).__init__(env)
+        self.enforce_not_on_barrier = True
+        self.env.check_collisions_for(self)
+
+    def on_collected(self):
+        # Force rerender of value func (even in static envs)
+        self.env._invalidate_value_map()
+
+        self.env.new_abs_reward += self.reward
+        self.pos = (self.env.random(), self.env.random())
+        self.env.check_collisions_for(self)
+
+
+class FootballPlayer():
+    def __init__(self, env, target):
+        super(FootballPlayer, self).__init__(env)
+        self.col = (200, 0, 100)
+        self.target = target
+        self.solid = True
+        self.movable = 1
+        self.elasticity = 1
+
+
+class WalkingFootballPlayer(FootballPlayer, WalkingChaser):
+    def __init__(self, env, target):
+        super(WalkingFootballPlayer, self).__init__(env, target)
+        self.target = target
+
+
+class FlyingFootballPlayer(FootballPlayer, FlyingChaser):
+    def __init__(self, env, target):
+        super(FlyingFootballPlayer, self).__init__(env, target)
+
 # Not a real entity. Is used in the config of RayObserver to reference the outer boundary of the environment.
+
+
 class Void():
     def __init__(self, env):
         self.col = (50, 50, 50)

columbus/humanPlayer.py

@@ -1,16 +1,18 @@
+import torch as th
 from time import sleep, time
 import numpy as np
 import pygame
+import yaml
 
 from columbus import env
 from columbus.observables import Observable, CnnObservable
 
 
 def main():
-    Env = chooseEnv()
+    env = chooseEnv()
-    env = Env(fps=30)
+    while True:
-    env.start_pos = [0.6, 0.3]
+        playEnv(env)
-    playEnv(env)
+        input('<again?>')
     env.close()
 
 
@@ -22,6 +24,33 @@ def getAvaibleEnvs():
             yield getattr(env, s)
 
 
+def loadConfigDefinedEnv(EnvClass):
+    p = input('[Path to config> ')
+    with open(p, 'r') as f:
+        docs = list([d for d in yaml.safe_load_all(
+            f) if d and 'name' in d and d['name'] not in ['SLURM']])
+    for i, doc in enumerate(docs):
+        name = doc['name']
+        print('['+str(i)+'] '+name)
+    ds = int(input('[0]> ') or '0')
+    doc = docs[ds]
+    cur = doc
+    path = 'params.task.env_args'
+    p = path.split('.')
+    while True:
+        try:
+            if len(p) == 0:
+                break
+            key = p.pop(0)
+            print(key)
+            cur = cur[key]
+        except Exception as e:
+            print('Unable to find key "'+key+'"')
+            path = input('[Path> ')
+    print(cur)
+    return EnvClass(fps=30, **cur)
+
+
 def chooseEnv():
     envs = list(getAvaibleEnvs())
     for i, Env in enumerate(envs):
@@ -35,14 +64,23 @@ def chooseEnv():
         if i < 0 or i >= len(envs):
             print(
                 '[!] That is a number, but not one that makes sense in this context...')
-        return envs[i]
+        if envs[i] in [env.ColumbusConfigDefined]:
+            return loadConfigDefinedEnv(envs[i])
+        Env = envs[i]
+        return Env(fps=30)
+
+
+def value_func(obs):
+    return obs[:, 0]
+    # return th.rand(obs.shape[0])-0.5
 
 
 def playEnv(env):
-    env.reset()
     done = False
+    env.reset()
     while not done:
         t1 = time()
+        # env.render(value_func=value_func)
         env.render()
         pos = (0.5, 0.5)
         pos = pygame.mouse.get_pos()

columbus/observables.py

@@ -3,29 +3,33 @@ import numpy as np
 import pygame
 import math
 from columbus import entities
+import torch as th
 
 
 class Observable():
     def __init__(self):
         self.obs = None
-        pass
 
     def _set_env(self, env):
         self.env = env
 
-    def get_observation_space():
+    def get_observation_space(self):
         print("[!] Using dummyObservable. Env won't output anything")
-        return spaces.Box(low=0, high=255,
+        return spaces.Box(low=0, high=1,
-                          shape=(1,), dtype=np.uint8)
+                          shape=(1,), dtype=np.float64)
 
     def get_observation(self):
-        return False
+        return np.array([0])
 
     def draw(self):
         pass
 
+    def reset(self):
+        pass
+
 
 class CnnObservable(Observable):
+    # Currently broken...
     def __init__(self, in_width=256, in_height=256, out_width=32, out_height=32, draw_width=128, draw_height=128, smooth_scaling=True):
         super(CnnObservable, self).__init__()
         self.in_width = in_width
@@ -41,11 +45,11 @@ class CnnObservable(Observable):
 
     def get_observation_space(self):
         return spaces.Box(low=0, high=255,
-                          shape=(self.out_width, self.out_height, 3), dtype=np.uint8)
+                          shape=(self.out_width, self.out_height, 3), dtype=np.float64)
 
     def get_observation(self):
         if not self.env._rendered:
-            self.env.render(dont_show=True)
+            self.env.render(mode='internal', dont_show=False)
         self.env._ensure_surface()
         x, y = self.env.agent.pos[0]*self.env.width - self.in_width / \
             2, self.env.agent.pos[1]*self.env.height - self.in_height/2
@@ -97,7 +101,7 @@ class RayObservable(Observable):
         self.include_rand = include_rand
 
     def get_observation_space(self):
-        return spaces.Box(low=0, high=self.num_steps,
+        return spaces.Box(low=0, high=1,
                           shape=(self.num_rays+self.include_rand, self.num_chans), dtype=np.uint8)
 
     def _get_ray_heads(self):
@@ -109,34 +113,56 @@ class RayObservable(Observable):
         for entity in entities_l:
             if isinstance(entity, entity_type) or (self.env.void_barrier and isinstance(entity, entities.Void) and entity_type == entities.Enemy):
                 if isinstance(entity, entities.Void):
-                    if 0 >= pos[0] or pos[0] >= self.env.width or 0 >= pos[1] or pos[1] >= self.env.width:
+                    if not self.env.torus_topology and (0 >= pos[0] or pos[0] >= self.env.width or 0 >= pos[1] or pos[1] >= self.env.width):
                         return True
                 else:
-                    if entity.shape != 'circle':
+                    if entity.shape == 'circle':
-                        raise Exception('Can only raycast circular entities!')
+                        sq_dist = (pos[0]-entity.pos[0]*self.env.width) ** 2 \
-                    sq_dist = (pos[0]-entity.pos[0]*self.env.width) ** 2 \
+                            + (pos[1]-entity.pos[1]*self.env.height)**2
-                        + (pos[1]-entity.pos[1]*self.env.height)**2
+                        if sq_dist < entity.radius**2:
-                    if sq_dist < entity.radius**2:
+                            return True
-                        return True
+                    elif entity.shape == 'rect':
+                        dot = entities.CircularEntity(self.env)
+                        dot.radius = 1
+                        dot.pos = pos[0]/self.env.width, pos[1]/self.env.height
+                        if sum(dot._get_crash_force_dir(entity)) != 0:
+                            return True
+                    else:
+                        raise Exception(
+                            'Can only raycast circular and rectangular entities!')
         return False
 
+    # Filter out entities, that we sure are out of range
+    # (so we have to do less work for the ray collisions)
     def _get_possible_entities(self):
         entities_l = []
         if entities.Void in self.chans or self.env.void_barrier:
             entities_l.append(entities.Void(self.env))
         for entity in self.env.entities:
-            sq_dist = ((self.env.agent.pos[0]-entity.pos[0])*self.env.width) ** 2 \
+            if entity.shape == 'rect':
-                + ((self.env.agent.pos[1]-entity.pos[1])*self.env.height) ** 2
+                x, y = entity.pos[0]+entity.width/self.env.width / \
-            if sq_dist <= (entity.radius + self.env.agent.radius + self.ray_len)**2:
+                    2, entity.pos[1]+entity.height/self.env.height/2
+                radius = (entity.width/2 + entity.height/2)*1.0
+            elif entity.shape == 'circle':
+                x, y = entity.pos[0], entity.pos[1]
+                radius = entity.radius
+            else:
+                raise Exception(
+                    'Can only raycast circular and rectangular entities!')
+            sq_dist = ((self.env.agent.pos[0]-x)*self.env.width) ** 2 \
+                + ((self.env.agent.pos[1]-y)*self.env.height) ** 2
+            if sq_dist <= (radius + self.env.agent.getQuasiRadius() + self.ray_len)**2:
                 entities_l.append(entity)  # cannot use yield here!
         return entities_l
 
+    # Ugly, inefficient ray casting
+    # Oh well, it works...
     def get_observation(self):
         entities = self._get_possible_entities()
         self.rays = np.zeros((self.num_rays+self.include_rand, self.num_chans))
         if self.include_rand:
             for c in range(self.num_chans):
-                self.rays[-1, c] = self.env.random()
+                self.rays[-1, c] = np.random.rand()
         for r, (hx, hy) in enumerate(self._get_ray_heads()):
             occ_dist = self.num_steps
             for c, entity_type in enumerate(self.chans):
@@ -147,8 +173,10 @@ class RayObservable(Observable):
                     rx, ry = sx + \
                         self.env.agent.pos[0]*self.env.width, sy + \
                         self.env.agent.pos[1]*self.env.height
+                    if self.env.torus_topology:
+                        rx, ry = rx % self.env.width, ry % self.env.height
                     if self._check_collision((rx, ry), entity_type, entities):
-                        self.rays[r, c] = self.num_steps-s
+                        self.rays[r, c] = (self.num_steps-s)/self.num_steps
                         if self.occlusion:
                             occ_dist = s
                         break
@@ -157,18 +185,25 @@ class RayObservable(Observable):
     def draw(self):
         for c, entity_type in enumerate(self.chans):
             for r, (hx, hy) in enumerate(self._get_ray_heads()):
-                s = self.num_steps - self.rays[r, c]
+                s = self.num_steps - self.rays[r, c]*self.num_steps
                 sx, sy = (s+1)*hx/self.num_steps, (s+1)*hy/self.num_steps
                 rx, ry = sx + \
                     self.env.agent.pos[0]*self.env.width, sy + \
                     self.env.agent.pos[1]*self.env.height
+                if self.env.torus_topology:
+                    rx, ry = rx % self.env.width, ry % self.env.height
                 # TODO: How stupid do I want to code?
+                # This instanciates an Object for every Ray-hit,
+                # just to get the color for the visual.
+                # But since this Code will not be executed during training,
+                # I don't think fixing this is an priority...
                 col = entity_type(self.env).col
                 col = int(col[0]/2), int(col[1]/2), int(col[2]/2)
                 pygame.draw.circle(self.env.screen, col, (rx, ry), 3, width=0)
 
 
 class StateObservable(Observable):
+    # Whitelists probably don't work...
     def __init__(self, coordsAgent=False, speedAgent=False, coordsRelativeToAgent=True, coordsRewards=True, rewardsWhitelist=None, coordsEnemys=True, enemysWhitelist=None, enemysNoBarriers=True, rewardsTimeouts=True, include_rand=True):
         super(StateObservable, self).__init__()
         self._entities = None
@@ -208,12 +243,15 @@ class StateObservable(Observable):
                     self._timeoutEntities.append(entity)
         return self._entities
 
+    def reset(self):
+        self._entities = None
+
     def get_observation_space(self):
-        self.env.reset()
+        self.reset()
         num = len(self.entities)*2+len(self._timeoutEntities) + \
-            self.speedAgent + self.include_rand
+            self.speedAgent*2 + self.include_rand
-        return spaces.Box(low=0-1*self.coordsRelativeToAgent, high=1,
+        return spaces.Box(low=0-1*(self.coordsRelativeToAgent or self.speedAgent), high=1,
-                          shape=(num,), dtype=np.float32)
+                          shape=(num,), dtype=np.float64)
 
     def get_observation(self):
         obs = []
@@ -233,10 +271,10 @@ class StateObservable(Observable):
         for entity in self._timeoutEntities:
             obs.append(entity.active)
         if self.speedAgent:
-            obs.append(self.env.speed[0])
+            obs.append(self.env.agent.speed[0])
-            obs.append(self.env.speed[1])
+            obs.append(self.env.agent.speed[1])
         if self.include_rand:
-            obs.append(self.env.random())
+            obs.append(np.random.rand())
         self.obs = obs
         return np.array(obs)
 
@@ -248,6 +286,77 @@ class StateObservable(Observable):
                                (0, self.env.height/2), 3, width=0)
             pygame.draw.circle(self.env.screen, self.env.agent.col,
                                (self.env.width/2, 0), 3, width=0)
+        for i in range(int(len(self.obs)/2) - self.speedAgent):
+            x, y = self.obs[i*2], self.obs[i*2+1]
+            col = self.entities[i].col
+            pygame.draw.circle(self.env.screen, col,
+                               (0, y*self.env.height+ofs[0]), 1, width=0)
+            pygame.draw.circle(self.env.screen, col,
+                               (x*self.env.width+ofs[1], 0), 1, width=0)
+
+
+class CompassObservable(Observable):
+    # Usefull for navigation close to an reward.
+    # Works like the StateObservable, but we assign a bigger range of possible input values to those, that are close to zero.
+    # I found that Agents without such an Observable often moved close to a reward and then just jiggled arround, adding a CompassObservable fixes this
+    def __init__(self, coordsRewards=True, rewardsWhitelist=None, coordsEnemys=False, enemysWhitelist=None, enemysNoBarriers=True):
+        super().__init__()
+        self._entities = None
+        self._timeoutEntities = []
+        self.coordRewards = coordsRewards
+        self.rewardsWhitelist = rewardsWhitelist
+        self.coordsEnemys = coordsEnemys
+        self.enemysWhitelist = enemysWhitelist
+        self.enemysNoBarriers = enemysNoBarriers
+
+    @property
+    def entities(self):
+        if not self._entities == None:
+            return self._entities
+        rewardsWhitelist = self.rewardsWhitelist or self.env.entities
+        enemysWhitelist = self.enemysWhitelist or self.env.entities
+        self._entities = []
+        if self.coordRewards:
+            for entity in rewardsWhitelist:
+                if isinstance(entity, entities.Reward):
+                    self._entities.append(entity)
+        if self.coordsEnemys:
+            for entity in enemysWhitelist:
+                if isinstance(entity, entities.Enemy):
+                    if not self.enemysNoBarriers or not isinstance(entity, entities.Barrier):
+                        self._entities.append(entity)
+        return self._entities
+
+    def get_observation_space(self):
+        self.reset()
+        num = len(self.entities)*2
+        return spaces.Box(low=-1, high=1,
+                          shape=(num,), dtype=np.float64)
+
+    def reset(self):
+        self._entities = None
+
+    def get_observation(self):
+        obs = []
+        for entity in self.entities:
+            dx, dy = entity.pos[0] - \
+                self.env.agent.pos[0], entity.pos[1] - self.env.agent.pos[1]
+            l = math.sqrt(dx**2 + dy**2)*2
+            x, y = math.tanh(dx/l), math.tanh(dy/l)
+            obs.append(x)
+            obs.append(y)
+
+        self.obs = obs
+        return np.array(obs)
+
+    def draw(self):
+        ofs = (0 + self.env.height/2,
+               0 + self.env.width/2)
+        if True:
+            pygame.draw.circle(self.env.screen, self.env.agent.col,
+                               (0, self.env.height/2), 3, width=0)
+            pygame.draw.circle(self.env.screen, self.env.agent.col,
+                               (self.env.width/2, 0), 3, width=0)
         for i in range(int(len(self.obs)/2)):
             x, y = self.obs[i*2], self.obs[i*2+1]
             col = self.entities[i].col
@@ -255,3 +364,43 @@ class StateObservable(Observable):
                                (0, y*self.env.height+ofs[0]), 1, width=0)
             pygame.draw.circle(self.env.screen, col,
                                (x*self.env.width+ofs[1], 0), 1, width=0)
+
+
+class CompositionalObservable(Observable):
+    # Used whenever you want to attach multiple Observables to an Env.
+    # We currently flatten the outputs of all attached Observables, so using a CNN though an CompositionalObservable would lead to problems.
+    def __init__(self, observables):
+        super().__init__()
+        self.observables = observables
+
+    def get_observation_space(self):
+        num = 0
+        for i, obs in enumerate(self.observables):
+            space = obs.get_observation_space()
+            num += math.prod(space.shape)
+            if not i:
+                low = space.low.reshape((-1))
+                high = space.high.reshape((-1))
+            else:
+                low = np.hstack((low, space.low.reshape((-1))))
+                high = np.hstack((high, space.high.reshape((-1))))
+        return spaces.Box(low=low, high=high,
+                          shape=(num,), dtype=np.float64)
+
+    def get_observation(self):
+        o = [obs.get_observation().reshape((-1))
+             for obs in self.observables]
+        o = np.hstack(o)
+        return o
+
+    def draw(self):
+        for obs in self.observables:
+            obs.draw()
+
+    def _set_env(self, env):
+        for obs in self.observables:
+            obs._set_env(env)
+
+    def reset(self):
+        for obs in self.observables:
+            obs.reset()

columbus/utils.py

@@ -0,0 +1,42 @@
+from columbus import entities, observables
+
+import random as random_dont_use
+
+
+def parseObs(obsConf):
+    # Parsing Observable Definitions
+    if type(obsConf) == list:
+        obs = []
+        for i, c in enumerate(obsConf):
+            obs.append(parseObs(c))
+        if len(obs) == 1:
+            return obs[0]
+        else:
+            return observables.CompositionalObservable(obs)
+
+    if obsConf['type'] == 'State':
+        conf = {k: v for k, v in obsConf.items() if k not in ['type']}
+        return observables.StateObservable(**conf)
+    elif obsConf['type'] == 'Compass':
+        conf = {k: v for k, v in obsConf.items() if k not in ['type']}
+        return observables.CompassObservable(**conf)
+    elif obsConf['type'] == 'RayCast':
+        chans = []
+        for chan in obsConf.get('chans', []):
+            chans.append(getattr(entities, chan))
+        conf = {k: v for k, v in obsConf.items() if k not in ['type', 'chans']}
+        return observables.RayObservable(chans=chans, **conf)
+    elif obsConf['type'] == 'CNN':
+        conf = {k: v for k, v in obsConf.items() if k not in ['type']}
+        return observables.CnnObservable(**conf)
+    elif obsConf['type'] == 'Dummy':
+        conf = {k: v for k, v in obsConf.items() if k not in ['type']}
+        return observables.Observable(**conf)
+    else:
+        raise Exception('Unknown Observable selected')
+
+
+def soft_int(num):
+    i = int(num)
+    r = num - i
+    return i + int(random_dont_use.random() < r)

configs/Env_A_Local_Optima.yaml

@@ -0,0 +1,78 @@
+name: "DEFAULT"
+
+params:
+  task:
+    task: columbus
+    env_name: ColumbusConfigDefined-v0
+    env_args:
+      observable:
+        - type: State
+          coordsAgent: True
+          speedAgent: True
+          coordsRelativeToAgent: False
+          coordsRewards: True
+          coordsEnemys: False
+          enemysNoBarriers: True
+          rewardsTimeouts: False
+          include_rand: True
+        - type: State
+          coordsAgent: False
+          speedAgent: False
+          coordsRelativeToAgent: True
+          coordsRewards: True
+          coordsEnemys: False
+          enemysNoBarriers: True
+          rewardsTimeouts: False
+          include_rand: True
+        - type: Compass
+        - type: RayCast
+          num_rays: 6
+          chans: [Enemy]
+      entities:
+        - type: RectBarrier
+          damage: 1 #1
+          width: 300
+          height: 120 # 360 - 5%(720)
+          pos: [0, 0]
+        - type: RectBarrier
+          damage: 1 #1
+          width: 300
+          height: 1000
+          pos: [0, 0.25]
+        - type: RectBarrier
+          damage: 1 #1
+          width: 250
+          height: 30
+          pos: [0.55, 0.6]
+        - type: RectBarrier
+          damage: 1 #1
+          width: 30
+          height: 120
+          pos: [0.856, 0.475]
+        - type: RectBarrier
+          num: 0
+          damage: 1 #1
+          width: 50
+          width_rand: 100
+          height: 25
+          height_rand: 100
+        - type: OnceReward
+          reward: 100
+          radius: 20
+          pos: [0.9, 0.8]
+      start_pos: [0.1, 0.21]
+      default_collision_elasticity: 0.8
+      start_score: 10
+      speed_fac: 0.01
+      acc_fac: 0.1
+      die_on_zero: False #True
+      agent_drag: 0.1 # 0.05
+      controll_type: ACC # SPEED
+      aux_reward_max: 1
+      aux_penalty_max: 0.01
+      void_damage: 5 #1
+      terminate_on_reward: True
+      agent_draw_path: True
+      clear_path_on_reset: False
+      max_steps: 450 # 1800
+---

configs/Env_B_Random_Gen.yaml

@@ -0,0 +1,54 @@
+name: "DEFAULT"
+
+params:
+  task:
+    task: columbus
+    num_envs: 8
+    env_args:
+      observable:
+        - type: State
+          coordsAgent: True
+          speedAgent: True
+          coordsRelativeToAgent: False
+          coordsRewards: True
+          coordsEnemys: False
+          enemysNoBarriers: True
+          rewardsTimeouts: False
+          include_rand: True
+        - type: State
+          coordsAgent: False
+          speedAgent: False
+          coordsRelativeToAgent: True
+          coordsRewards: True
+          coordsEnemys: False
+          enemysNoBarriers: True
+          rewardsTimeouts: False
+          include_rand: True
+        - type: Compass
+        - type: RayCast
+          num_rays: 8
+          chans: [Enemy]
+      entities:
+        - type: CircleBarrier
+          num: 8
+          num_rand: 6
+          damage: 20 #20
+          radius: 25
+          radius_rand: 75
+        - type: TeleportingReward
+          num: 1
+          reward: 100 #100
+          radius: 20
+      default_collision_elasticity: 0.8
+      start_score: 50
+      speed_fac: 0.01
+      acc_fac: 0.1
+      die_on_zero: True
+      agent_drag: 0.07 # 0.05
+      controll_type: ACC # SPEED
+      aux_reward_max: 1
+      aux_penalty_max: 0.1
+      void_damage: 5 #1
+      #master_seed: 3.14
+      max_steps: 900 # 30 sec
+---

configs/Env_C_Passage.yaml

@@ -0,0 +1,67 @@
+name: "DEFAULT"
+
+params:
+  task:
+    task: columbus
+    env_name: ColumbusConfigDefined-v0
+    env_args:
+      observable:
+        - type: State
+          coordsAgent: True
+          speedAgent: True
+          coordsRelativeToAgent: False
+          coordsRewards: True
+          coordsEnemys: False
+          enemysNoBarriers: True
+          rewardsTimeouts: False
+          include_rand: True
+        - type: State
+          coordsAgent: False
+          speedAgent: False
+          coordsRelativeToAgent: True
+          coordsRewards: True
+          coordsEnemys: False
+          enemysNoBarriers: True
+          rewardsTimeouts: False
+          include_rand: True
+        - type: RayCast
+          num_rays: 6
+          chans: [Enemy]
+      entities:
+        - type: RectBarrier
+          damage: 10 #1
+          width: 25
+          height: 120 # 360 - 5%(720)
+          pos: [0.45, 0]
+        - type: RectBarrier
+          damage: 10 #1
+          width: 25
+          height: 1000
+          pos: [0.45, 0.25]
+        - type: RectBarrier
+          damage: 10 #1
+          width: 25
+          height: 520 # 360 - 5%(720)
+          pos: [0.55, 0]
+        - type: RectBarrier
+          damage: 10 #1
+          width: 25
+          height: 200
+          pos: [0.55, 0.80]
+        - type: LoopReward
+          num: 1
+          reward: 100 #25
+          radius: 20
+          loop: [[0.125, 0.5, 0.1, 0.5], [0.875, 0.5, 0.1, 0.5]]
+      default_collision_elasticity: 0.8
+      start_score: 10
+      speed_fac: 0.01
+      acc_fac: 0.1
+      die_on_zero: False #True
+      agent_drag: 0.1 # 0.05
+      controll_type: ACC # SPEED
+      aux_reward_max: 1
+      aux_penalty_max: 0.01
+      void_damage: 5 #1
+      agent_draw_path: True
+---

configs/Example_Units.yaml

@@ -0,0 +1,92 @@
+name: "DEFAULT"
+
+# Supported Units:
+# px: Pixels
+# em: 1em = Full Width / Height
+# ct: 100ct = Full Width / Height
+# rx: pixels relative to width
+# ry: pixels relative to height
+# au: 1au = 36px (https://knowyourmeme.com/memes/absolute-unit)
+#
+# When no unit is given, we use the folowing defaults
+# (compatible with legacy behavior)
+# pos: em
+# all other: px
+#
+# ct is the recommendet unit.
+# If you need a unit, that is not responsive in regards to width/height, use au / px.
+
+params:
+  task:
+    task: columbus
+    env_name: ColumbusConfigDefined-v0
+    env_args:
+      observable:
+        - type: State
+          coordsAgent: True
+          speedAgent: True
+          coordsRelativeToAgent: False
+          coordsRewards: True
+          coordsEnemys: False
+          enemysNoBarriers: True
+          rewardsTimeouts: False
+          include_rand: True
+        - type: State
+          coordsAgent: False
+          speedAgent: False
+          coordsRelativeToAgent: True
+          coordsRewards: True
+          coordsEnemys: False
+          enemysNoBarriers: True
+          rewardsTimeouts: False
+          include_rand: True
+        - type: Compass
+        - type: RayCast
+          num_rays: 6
+          chans: [Enemy]
+      entities:
+        - type: RectBarrier
+          num: 1
+          width: 50ct
+          height: 50ct
+          pos: [0ct, 0ct]
+        - type: RectBarrier
+          num: 1
+          width: 50ct
+          height: 50ct
+          pos: [50ct, 50ct]
+        - type: RectBarrier
+          num: 1
+          width: 25rx
+          height: 25ry
+          pos: [0.75em, 30px]
+        - type: RectBarrier
+          num: 1
+          width: 25ry
+          height: 25rx
+          pos: [0.75em, 60px]
+        - type: RectBarrier
+          num: 1
+          width: 20 # defaults to rx (px scaled from x-axis)
+          height: 10 # defaults to ry (px scaled from y-axis)
+          pos: [0.75em, 90px]
+        - type: OnceReward
+          reward: 100
+          radius: 1au
+          pos: [0.3, 0.8] # defaults to em
+      start_pos: [90ct, 20ct]
+      default_collision_elasticity: 0.8
+      start_score: 10
+      speed_fac: 0.01
+      acc_fac: 0.1
+      die_on_zero: False #True
+      agent_drag: 0.1 # 0.05
+      controll_type: ACC # SPEED
+      aux_reward_max: 1
+      aux_penalty_max: 0.01
+      void_damage: 5 #1
+      terminate_on_reward: True
+      agent_draw_path: True
+      clear_path_on_reset: False
+      max_steps: 450 # 1800
+---

configs/Pong.yaml

@@ -0,0 +1,108 @@
+name: "DEFAULT"
+
+params:
+  task:
+    task: columbus
+    env_name: Columbus-v1
+    env_args:
+      observable:
+        - type: State
+          coordsAgent: True
+          speedAgent: True
+          coordsRelativeToAgent: False
+          coordsRewards: True
+          coordsEnemys: False
+          enemysNoBarriers: True
+          rewardsTimeouts: False
+          include_rand: True
+        - type: State
+          coordsAgent: False
+          speedAgent: False
+          coordsRelativeToAgent: True
+          coordsRewards: True
+          coordsEnemys: False
+          enemysNoBarriers: True
+          rewardsTimeouts: False
+          include_rand: True
+        - type: Compass
+        - type: RayCast
+          num_rays: 6
+          chans: [Enemy]
+      entities:
+        - type: Ball
+          radius: 16px
+          pos: [0.8, 0.5]
+          speed: [-0.2, -0.1]
+          speed_rand: [0, 0.2]
+          solid: True
+          collision_elasticity: 3
+          elasticity: 1
+          movable: 1
+          collision_changes_speed: True
+          crash_conservation_of_energy: False
+          min_speed: 0.2
+          max_speed: 0.6
+          draw_path: True
+          draw_path_width: 32
+          draw_path_harm: True
+          drag: 0.00001
+        - type: RectGoal # Good
+          height: 1em
+          width: 10ct
+          pos: [97ct, 0ct]
+          skip_agent_col_check: True
+          col: [0, 255, 0]
+          reward: 30
+          solid: True
+          elasticity: 0.6
+          void_collidable: False
+        - type: Goal # Top
+          radius: 7ct
+          pos: [100ct, 0ct]
+          skip_agent_col_check: True
+          col: [0, 255, 0]
+          reward: 30
+          solid: True
+          elasticity: 0.7
+          void_collidable: False
+        - type: Goal # Bottom
+          radius: 7ct
+          pos: [100ct, 100ct]
+          skip_agent_col_check: True
+          col: [0, 255, 0]
+          reward: 30
+          solid: True
+          elasticity: 0.7
+          void_collidable: False
+        - type: RectGoal # Bad
+          height: 1em
+          width: 3ct
+          pos: [0ct, 0ct]
+          skip_agent_col_check: True
+          col: [255, 0, 0]
+          reward: -45
+          solid: True
+          elasticity: 1000
+          void_collidable: False
+      agent_cls: PongAgent
+      agent_attrs:
+        height: 100
+        width: 30
+        movable: False
+        solid: True
+        elasticity: 0.9
+      exception_for_unsupported_collision: False
+      start_pos: [0.05, 0.5]
+      start_score: 0
+      speed_fac: 0.05
+      acc_fac: 0.1
+      die_on_zero: False #True
+      agent_drag: 0
+      controll_type: SPEED
+      aux_reward_max: 0
+      aux_penalty_max: 0
+      void_damage: 0
+      terminate_on_reward: False
+      agent_draw_path: False
+      clear_path_on_reset: False
+---

configs/example.yaml

@@ -0,0 +1,49 @@
+name: Example
+params:
+ task:
+  env_args:
+      observable:
+        - type: State
+          coordsAgent: True
+          speedAgent: True
+          coordsRelativeToAgent: False
+          coordsRewards: True
+          coordsEnemys: False
+          enemysNoBarriers: True
+          rewardsTimeouts: False
+          include_rand: True
+        - type: State
+          coordsAgent: False
+          speedAgent: False
+          coordsRelativeToAgent: True
+          coordsRewards: True
+          coordsEnemys: False
+          enemysNoBarriers: True
+          rewardsTimeouts: False
+          include_rand: True
+        - type: Compass
+        - type: RayCast
+          num_rays: 8
+          chans: [Enemy]
+      entities:
+        - type: CircleBarrier
+          num: 8
+          num_rand: 6
+          damage: 20
+          radius: 25
+          radius_rand: 75
+        - type: TeleportingReward
+          num: 1
+          reward: 100
+          radius: 20
+      default_collision_elasticity: 0.8
+      start_score: 50
+      speed_fac: 0.01
+      acc_fac: 0.1
+      die_on_zero: True
+      agent_drag: 0.07
+      controll_type: ACC
+      aux_reward_max: 1
+      aux_penalty_max: 0.1
+      void_damage: 5
+---

icon.svg

@@ -8,6 +8,11 @@
    version="1.1"
    id="svg5"
    xml:space="preserve"
+   inkscape:export-filename="columbus.svg"
+   inkscape:export-xdpi="96"
+   inkscape:export-ydpi="96"
+   sodipodi:docname="columbus.svg"
+   inkscape:version="1.2 (dc2aedaf03, 2022-05-15)"
    xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
    xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
    xmlns="http://www.w3.org/2000/svg"
@@ -21,23 +26,32 @@
      inkscape:pagecheckerboard="0"
      inkscape:deskcolor="#505050"
      inkscape:document-units="mm"
-     showgrid="false" /><defs
+     showgrid="false"
+     inkscape:zoom="1.4024329"
+     inkscape:cx="16.043548"
+     inkscape:cy="166.8529"
+     inkscape:window-width="1920"
+     inkscape:window-height="1050"
+     inkscape:window-x="0"
+     inkscape:window-y="0"
+     inkscape:window-maximized="1"
+     inkscape:current-layer="g3514" /><defs
      id="defs2" /><g
      inkscape:label="Layer 1"
      inkscape:groupmode="layer"
      id="layer1"
      transform="translate(-50.719907,-147.38589)"><path
        style="line-height:normal;text-indent:0;text-align:start;text-decoration-line:none;text-decoration-style:solid;text-decoration-color:#000000;text-transform:none;isolation:auto;mix-blend-mode:normal;stroke-width:2.53468"
-       d="m 74.860518,149.35966 c -0.485108,0.0261 -0.952504,0.19102 -1.346548,0.47517 -24.095861,16.78439 -30.037141,49.999 -13.252643,74.09499 9.803749,14.07433 25.215631,21.89775 41.139133,22.67355 v 7.87631 c -10.184721,0.68784 -21.203241,5.10896 -21.203241,5.10896 -2.593164,1.02902 -1.864039,4.88103 0.925813,4.89113 h 22.386418 c 0.27372,0.0452 0.55302,0.0452 0.82674,0 h 22.41116 c 2.78963,-0.0105 3.51837,-3.8622 0.92541,-4.89113 0,0 -11.01854,-4.42112 -21.20326,-5.10896 v -7.86645 c 9.68226,-0.44834 19.35645,-3.48901 27.88649,-9.43075 2.9157,-1.90396 -0.0991,-6.23299 -2.89609,-4.15854 -8.29383,5.77725 -17.75973,8.57401 -27.14389,8.64367 -0.13897,-0.0245 -0.27974,-0.0373 -0.42085,-0.0384 -0.0694,1.5e-4 -0.13874,0.003 -0.20788,0.009 -15.118965,-0.0861 -29.955861,-7.24047 -39.267824,-20.60886 -15.218446,-21.84776 -9.857594,-51.82198 11.990241,-67.04043 2.022521,-1.36404 1.139716,-4.51789 -1.29704,-4.63372 -0.08409,-0.004 -0.168339,-0.004 -0.25243,0 z"
+       d="m 74.860518,149.35966 c -0.485108,0.0261 -0.952504,0.19102 -1.346548,0.47517 -24.095861,16.78439 -30.037141,49.999 -13.252643,74.09499 9.803749,14.07433 25.215631,21.89775 41.139133,22.67355 v 11.58048 c -10.184721,0.68784 -21.203241,1.40479 -21.203241,1.40479 -2.593164,1.02902 -1.864039,4.88103 0.925813,4.89113 h 22.386418 c 0.27372,0.0452 0.55302,0.0452 0.82674,0 h 22.41116 c 2.78963,-0.0105 3.51837,-3.8622 0.92541,-4.89113 0,0 -11.01854,-0.71695 -21.20326,-1.40479 v -11.57062 c 9.68226,-0.44834 19.35645,-3.48901 27.88649,-9.43075 2.9157,-1.90396 -0.0991,-6.23299 -2.89609,-4.15854 -8.29383,5.77725 -17.75973,8.57401 -27.14389,8.64367 -0.13897,-0.0245 -0.27974,-0.0373 -0.42085,-0.0384 -0.0694,1.5e-4 -0.13874,0.003 -0.20788,0.009 -15.118965,-0.0861 -29.955861,-7.24047 -39.267824,-20.60886 -15.218446,-21.84776 -9.857594,-51.82198 11.990241,-67.04043 2.022521,-1.36404 1.139716,-4.51789 -1.29704,-4.63372 -0.08409,-0.004 -0.168339,-0.004 -0.25243,0 z"
        id="path219"
        sodipodi:nodetypes="ccscccccccccccccccscccc" /><path
-       d="m 104.02542,235.41497 v 0 c 23.29851,0 42.1806,-18.88209 42.1806,-42.18061 0,-23.2985 -18.88209,-42.1806 -42.1806,-42.1806 -23.298512,0 -42.180599,18.8821 -42.180599,42.1806 0,23.29852 18.882087,42.18061 42.180599,42.18061 z m 0,3.66787 v 0 c -25.319587,0 -45.848477,-20.52889 -45.848477,-45.84848 0,-25.31958 20.52889,-45.84847 45.848477,-45.84847 9.65374,0 18.61105,2.9843 25.99961,8.0806 11.98999,8.27016 19.84886,22.10203 19.84886,37.76787 0,25.31959 -20.52889,45.84848 -45.84847,45.84848 z m -23.289155,-22.70902 27.059955,-19.20023 1.07604,1.48774 -31.345387,22.25055 -0.224565,0.31813 -1.459665,-1.11346 0.336845,-0.46785 -0.0094,-0.0187 0.04679,-0.0375 22.09148,-31.11145 1.459673,1.11346 z m 50.891815,-50.62047 -22.36284,32.70212 -10.451584,-10.46093 z"
+       d="m 104.02542,235.41497 v 0 c 23.29851,0 42.1806,-18.88209 42.1806,-42.18061 0,-23.2985 -18.88209,-42.1806 -42.1806,-42.1806 -23.298512,0 -42.180599,18.8821 -42.180599,42.1806 0,23.29852 18.882087,42.18061 42.180599,42.18061 z m 0,3.66787 v 0 c -25.319587,0 -45.848477,-20.52889 -45.848477,-45.84848 0,-25.31958 20.52889,-45.84847 45.848477,-45.84847 9.65374,0 18.61105,2.9843 25.99961,8.0806 11.98999,8.27016 19.84886,22.10203 19.84886,37.76787 0,25.31959 -20.52889,45.84848 -45.84847,45.84848 z m -23.289155,-22.70902 26.893815,-18.95737 1.27696,1.32365 -31.693278,22.43668 -0.06366,0.0617 -1.30746,-1.12193 0.336845,-0.46785 2.158205,-2.82644 -2.365497,3.25476 22.552413,-31.74443 1.226419,1.21459 z m 51.123965,-50.78018 -22.59499,32.86183 -10.451584,-10.46093 z"
        id="path326"
        style="stroke-width:0.0935682"
        sodipodi:nodetypes="ssssssssssssscccccccccccccccc" /><g
        id="g3514"
        transform="matrix(0.1421688,0,0,0.1421688,70.795973,160.65622)"
-       style="fill:#000000;fill-opacity:0.1"><g
+       style="fill:#000000;fill-opacity:0.2"><g
          id="g3456"
          style="fill:#000000;fill-opacity:0.1">
 	<path
@@ -53,58 +67,59 @@
    id="path3450"
    style="fill:#000000;fill-opacity:0.1" />
 	<path
-   d="M 359.5,240 C 343.964,240 330.946,229.039 327.755,214.446 358.881,210.384 383,183.715 383,151.5 c 0,-4.143 -3.358,-7.5 -7.5,-7.5 -4.142,0 -7.5,3.357 -7.5,7.5 0,26.743 -21.757,48.5 -48.5,48.5 -13.143,0 -25.447,-5.171 -34.646,-14.561 -2.898,-2.958 -7.647,-3.007 -10.606,-0.108 -2.959,2.899 -3.008,7.647 -0.109,10.606 10.402,10.617 23.839,17.103 38.395,18.677 3.444,22.829 23.192,40.386 46.966,40.386 4.142,0 7.5,-3.357 7.5,-7.5 0,-4.143 -3.358,-7.5 -7.5,-7.5 z"
+   d="m 355.7779,243.7221 c -15.536,0 -24.8319,-14.6831 -28.0229,-29.2761 C 358.881,210.384 383,183.715 383,151.5 c 0,-4.143 -3.358,-7.5 -7.5,-7.5 -4.142,0 -7.5,3.357 -7.5,7.5 0,26.743 -21.757,48.5 -48.5,48.5 -13.143,0 -25.447,-5.171 -34.646,-14.561 -2.898,-2.958 -7.647,-3.007 -10.606,-0.108 -2.959,2.899 -3.008,7.647 -0.109,10.606 10.402,10.617 23.839,17.103 38.395,18.677 3.444,22.829 19.4699,44.1081 43.2439,44.1081 4.142,0 7.5,-3.357 7.5,-7.5 0,-4.143 -3.358,-7.5 -7.5,-7.5 z"
    id="path3452"
-   style="fill:#000000;fill-opacity:0.1" />
+   style="fill:#000000;fill-opacity:0.1"
+   sodipodi:nodetypes="scsssscsccsss" />
 	<path
    d="m 335.5,328 c -2.89,0 -5.73,0.212 -8.519,0.588 0.006,-0.363 0.019,-0.726 0.019,-1.088 0,-35.014 -28.486,-63.5 -63.5,-63.5 -4.142,0 -7.5,3.357 -7.5,7.5 0,4.143 3.358,7.5 7.5,7.5 26.743,0 48.5,21.757 48.5,48.5 0,1.714 -0.091,3.434 -0.269,5.133 C 288.462,342.063 272,364.889 272,391.5 c 0,4.143 3.358,7.5 7.5,7.5 4.142,0 7.5,-3.357 7.5,-7.5 0,-26.743 21.757,-48.5 48.5,-48.5 4.142,0 7.5,-3.357 7.5,-7.5 0,-4.143 -3.358,-7.5 -7.5,-7.5 z"
    id="path3454"
    style="fill:#000000;fill-opacity:0.1" />
 </g><g
          id="g3458"
-         style="fill:#000000;fill-opacity:0.1">
+         style="fill:#000000;fill-opacity:0.2">
 </g><g
          id="g3460"
-         style="fill:#000000;fill-opacity:0.1">
+         style="fill:#000000;fill-opacity:0.2">
 </g><g
          id="g3462"
-         style="fill:#000000;fill-opacity:0.1">
+         style="fill:#000000;fill-opacity:0.2">
 </g><g
          id="g3464"
-         style="fill:#000000;fill-opacity:0.1">
+         style="fill:#000000;fill-opacity:0.2">
 </g><g
          id="g3466"
-         style="fill:#000000;fill-opacity:0.1">
+         style="fill:#000000;fill-opacity:0.2">
 </g><g
          id="g3468"
-         style="fill:#000000;fill-opacity:0.1">
+         style="fill:#000000;fill-opacity:0.2">
 </g><g
          id="g3470"
-         style="fill:#000000;fill-opacity:0.1">
+         style="fill:#000000;fill-opacity:0.2">
 </g><g
          id="g3472"
-         style="fill:#000000;fill-opacity:0.1">
+         style="fill:#000000;fill-opacity:0.2">
 </g><g
          id="g3474"
-         style="fill:#000000;fill-opacity:0.1">
+         style="fill:#000000;fill-opacity:0.2">
 </g><g
          id="g3476"
-         style="fill:#000000;fill-opacity:0.1">
+         style="fill:#000000;fill-opacity:0.2">
 </g><g
          id="g3478"
-         style="fill:#000000;fill-opacity:0.1">
+         style="fill:#000000;fill-opacity:0.2">
 </g><g
          id="g3480"
-         style="fill:#000000;fill-opacity:0.1">
+         style="fill:#000000;fill-opacity:0.2">
 </g><g
          id="g3482"
-         style="fill:#000000;fill-opacity:0.1">
+         style="fill:#000000;fill-opacity:0.2">
 </g><g
          id="g3484"
-         style="fill:#000000;fill-opacity:0.1">
+         style="fill:#000000;fill-opacity:0.2">
 </g><g
          id="g3486"
-         style="fill:#000000;fill-opacity:0.1">
+         style="fill:#000000;fill-opacity:0.2">
 </g></g></g><style
      type="text/css"
      id="style139">

requirements.txt

@@ -0,0 +1,2 @@
+gym
+pygame