dodox/fancy_gym
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

Change .gitignore; push docs/build to GitHub

Browse Source
This commit is contained in:
dodox
2024-01-23 17:16:12 +01:00
parent 91b10cc90a
commit 4b862185e6
130 changed files with 27380 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/build/html/_sources/api.rst.txt
Vendored
+8
View File
@@ -0,0 +1,8 @@
API
===
.. autosummary::
:toctree: generated
fancy_gym.register
fancy_gym.upgrade
docs/build/html/_sources/envs/dmc.md.txt
Vendored
+79
View File
@@ -0,0 +1,79 @@
# DeepMind Control (DMC)
<!-- TODO: Add Vid -->
These are the Environment Wrappers for selected
[DeepMind Control](https://github.com/google-deepmind/dm_control)
environments in order to use our Motion Primitive gym interface with them.
## Step-Based Environments
When installing fancy_gym with the optional dmc extra (e.g. `pip install fancy_gym[dmc]`), all regular dmc tasks are avaible via [Shimmy](https://github.com/Farama-Foundation/Shimmy).
| Name | Description | Action Dim | Observation Dim |
| --------------------------------------- | ----------------------------------------------------------------------------------------------------------------------- | ---------- | --------------- |
| `dm_control/acrobot-swingup-v0` | Underactuated double pendulum (Acrobot) with torque applied to the second joint to swing up and balance. | 1 | 6 |
| `dm_control/acrobot-swingup_sparse-v0` | Similar to `acrobot-swingup-v0`, but with sparse rewards for achieving the swingup task. | 1 | 6 |
| `dm_control/ball_in_cup-catch-v0` | Planar ball-in-cup task where a receptacle must swing to catch a ball. Sparse reward for catching. | 2 | 8 |
| `dm_control/cartpole-balance-v0` | Cart-pole task where the goal is to balance an unactuated pole by moving a cart, starting near upright. | 1 | 5 |
| `dm_control/cartpole-balance_sparse-v0` | Similar to `cartpole-balance-v0`, but with sparse rewards. | 1 | 5 |
| `dm_control/cartpole-swingup-v0` | Cart-pole task with the pole starting downward, requiring swinging up and balancing. | 1 | 5 |
| `dm_control/cartpole-swingup_sparse-v0` | Similar to `cartpole-swingup-v0`, but with sparse rewards for the swingup task. | 1 | 5 |
| `dm_control/cartpole-two_poles-v0` | Extension of the Cart-pole domain with two serially connected poles, increasing the balancing challenge. | 1 | 8 |
| `dm_control/cartpole-three_poles-v0` | Extension of the Cart-pole domain with three serially connected poles, further increasing the challenge. | 1 | 11 |
| `dm_control/cheetah-run-v0` | Planar bipedal cheetah robot tasked with running. The reward is proportional to forward velocity up to a maximum speed. | 6 | 17 |
| `dm_control/dog-stand-v0` | Dog robot task focusing on achieving a standing posture. | 38 | 223 |
| `dm_control/dog-walk-v0` | Dog robot tasked with walking, requiring coordination of movements. | 38 | 223 |
| `dm_control/dog-trot-v0` | Dog robot performing a trotting gait. | 38 | 223 |
| `dm_control/dog-run-v0` | Dog robot tasked with running, combining speed with stability. | 38 | 223 |
| `dm_control/dog-fetch-v0` | Dog robot playing fetch, involving locomotion and object interaction. | 38 | 232 |
| `dm_control/finger-spin-v0` | Finger robot required to rotate an unactuated body on a hinge. | 2 | 9 |
| `dm_control/finger-turn_easy-v0` | Finger robot task to align the tip of a free body with a target, easier version with a larger target. | 2 | 12 |
| `dm_control/finger-turn_hard-v0` | Similar to `finger-turn_easy-v0`, but with a smaller target for increased difficulty. | 2 | 12 |
| `dm_control/fish-upright-v0` | Fish robot task focused on righting itself in a fluid environment. | 5 | 21 |
| `dm_control/fish-swim-v0` | Fish robot swimming to a target, incorporating fluid dynamics. | 5 | 24 |
| `dm_control/hopper-stand-v0` | One-legged hopper robot tasked with achieving a minimal torso height. | 4 | 15 |
| `dm_control/hopper-hop-v0` | One-legged hopper robot required to hop, combining height and forward velocity. | 4 | 15 |
| `dm_control/humanoid-stand-v0` | Simplified humanoid robot maintaining a standing posture. | 21 | 67 |
| `dm_control/humanoid-walk-v0` | Simplified humanoid robot walking at a specified speed. | 21 | 67 |
| `dm_control/humanoid-run-v0` | Simplified humanoid robot running, aiming for a high horizontal speed. | 21 | 67 |
| `dm_control/humanoid-run_pure_state-v0` | Simplified humanoid robot running with a focus on pure state control. | 21 | 55 |
| `dm_control/humanoid_CMU-stand-v0` | Advanced humanoid robot (CMU model) maintaining a standing posture. | 56 | 137 |
| `dm_control/humanoid_CMU-walk-v0` | Advanced humanoid robot (CMU model) walking. | 56 | 137 |
| `dm_control/humanoid_CMU-run-v0` | Advanced humanoid robot (CMU model) running. | 56 | 137 |
| `dm_control/lqr-lqr_2_1-v0` | Linear quadratic regulator (LQR) task with 2 masses and 1 actuator, focusing on position and control optimization. | 1 | 4 |
| `dm_control/lqr-lqr_6_2-v0` | Linear quadratic regulator (LQR) task with 6 masses and 2 actuators, more complex control optimization challenge. | 2 | 12 |
| `dm_control/manipulator-bring_ball-v0` | Planar manipulator robot bringing a ball to a target location, with object initialization variations. | 5 | 44 |
| `dm_control/manipulator-bring_peg-v0` | Planar manipulator task of bringing a peg to a target peg. | 5 | 44 |
| `dm_control/manipulator-insert_ball-v0` | Planar manipulator task requiring inserting a ball into a basket. | 5 | 44 |
| `dm_control/manipulator-insert_peg-v0` | Planar manipulator challenge of inserting a peg into a slot. | 5 | 44 |
| `dm_control/pendulum-swingup-v0` | Classic inverted pendulum task with a torque-limited actuator, requiring multiple swings to swing up and balance. | 1 | 3 |
| `dm_control/point_mass-easy-v0` | Planar point mass in an easy task, with actuators corresponding to global x and y axes | 2 | 4 |
| `dm_control/point_mass-hard-v0` | Planar point mass in a hard task, with randomized control gains per episode, challenging memoryless agents. | 2 | 4 |
| `dm_control/quadruped-walk-v0` | Four-legged robot (Quadruped) tasked with walking. | 12 | 78 |
| `dm_control/quadruped-run-v0` | Quadruped robot required to run, balancing speed and stability. | 12 | 78 |
| `dm_control/quadruped-escape-v0` | Quadruped robot in an escape task, combining locomotion with environmental interaction. | 12 | 101 |
| `dm_control/quadruped-fetch-v0` | Quadruped robot playing fetch, involving complex movements and object interaction. | 12 | 90 |
| `dm_control/reacher-easy-v0` | Two-link planar reacher with a randomized target, easier version with a larger target sphere. | 2 | 6 |
| `dm_control/reacher-hard-v0` | Similar to `reacher-easy-v0`, but with a smaller target sphere for increased difficulty. | 2 | 6 |
| `dm_control/stacker-stack_2-v0` | Stacker task requiring stacking of 2 boxes, with rewards for correct placement and gripper positioning. | 5 | 49 |
| `dm_control/stacker-stack_4-v0` | More complex stacker task with 4 boxes, increasing the stacking challenge. | 5 | 63 |
| `dm_control/swimmer-swimmer6-v0` | Six-link planar swimmer in fluid dynamics, rewarded for positioning the nose inside a target. | 5 | 25 |
| `dm_control/swimmer-swimmer15-v0` | Fifteen-link planar swimmer, a more complex version of the swimmer task with extended dynamics. | 14 | 61 |
| `dm_control/walker-stand-v0` | Planar walker robot tasked with maintaining an upright torso and minimal height. | 6 | 24 |
| `dm_control/walker-walk-v0` | Planar walker robot walking task, focusing on forward velocity and stability. | 6 | 24 |
| `dm_control/walker-run-v0` | Planar walker robot running task, aiming for high speed and balance. | 6 | 24 |
## MP Environments
[//]: <> (These environments are wrapped-versions of their Deep Mind Control Suite &#40;DMC&#41; counterparts. Given most task can be)
[//]: <> (solved in shorter horizon lengths than the original 1000 steps, we often shorten the episodes for those task.)
| Name | Description | Trajectory Horizon | Action Dimension | Context Dimension |
| ---------------------------------------- | ------------------------------------------------------------------------------ | ------------------ | ---------------- | ----------------- |
| `dm_control_ProDMP/ball_in_cup-catch-v0` | A ProMP wrapped version of the "catch" task for the "ball_in_cup" environment. | 1000 | 10 | 2 |
| `dm_control_DMP/ball_in_cup-catch-v0` | A DMP wrapped version of the "catch" task for the "ball_in_cup" environment. | 1000 | 10 | 2 |
| `dm_control_ProDMP/reacher-easy-v0` | A ProMP wrapped version of the "easy" task for the "reacher" environment. | 1000 | 10 | 4 |
| `dm_control_DMP/reacher-easy-v0` | A DMP wrapped version of the "easy" task for the "reacher" environment. | 1000 | 10 | 4 |
| `dm_control_ProDMP/reacher-hard-v0` | A ProMP wrapped version of the "hard" task for the "reacher" environment. | 1000 | 10 | 4 |
| `dm_control_DMP/reacher-hard-v0` | A DMP wrapped version of the "hard" task for the "reacher" environment. | 1000 | 10 | 4 |
docs/build/html/_sources/envs/fancy/airhockey.rst.txt
Vendored
+18
View File
@@ -0,0 +1,18 @@
AirHockey
=========
Fancy Gym provides access to a range of environments for the Robot Air Hockey Challenge, facilitating research in robot learning. The challenge aims to close the gap between simulated learning and real-world application by focusing on various aspects of robotic operation, such as dealing with disturbances, observation noise, safety, and actuator limitations.
The environments available through Fancy Gym allow for the development of agents capable of performing tasks with different levels of complexity. These tasks include hitting and defending in air hockey with both three degrees of freedom (3 DoF) and seven degrees of freedom (7 DoF) configurations. The 7 DoF tasks are based on the KUKA iiwa14 robot model, which is used in the simulations to represent a higher level of control complexity akin to real-world settings.
Participants in the challenge are required to develop strategies that enable their robots to react and adapt within these dynamic environments. The final phase of the challenge involves a tournament where the developed agents will be tested in a comprehensive game scenario, both in simulation and, for the top teams, on actual robotic systems.
For detailed information on the challenge, including rules, stages, and submission requirements, please visit the `official Robot Air Hockey Challenge website <https://air-hockey-challenge.robot-learning.net/>`_.
The available environments are as follows:
- fancy/AirHockey-7dof-hit-v0
- fancy/AirHockey-7dof-defend-v0
- fancy/AirHockey-3dof-hit-v0
- fancy/AirHockey-3dof-defend-v0
- fancy/AirHockey-7dof-hit-airhockit2023-v0
- fancy/AirHockey-7dof-defend-airhockit2023-v0
docs/build/html/_sources/envs/fancy/classic_control.md.txt
Vendored
+22
View File
@@ -0,0 +1,22 @@
# Classic Control
Classic Control environments provide a foundational platform for exploring and experimenting with RL algorithms. These environments are designed to be simple, allowing researchers and practitioners to focus on the fundamental principles of control without the complexities of high-dimensional and physics-based simulations.
## Step-based Environments
| Name | Description | Horizon | Action Dimension | Observation Dimension |
| ---------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------- | ---------------- | --------------------- |
| `fancy/SimpleReacher-v0` | Simple reaching task (2 links) without any physics simulation. Provides no reward until 150 time steps. This allows the agent to explore the space, but requires precise actions towards the end of the trajectory. | 200 | 2 | 9 |
| `fancy/LongSimpleReacher-v0` | Simple reaching task (5 links) without any physics simulation. Provides no reward until 150 time steps. This allows the agent to explore the space, but requires precise actions towards the end of the trajectory. | 200 | 5 | 18 |
| `fancy/ViaPointReacher-v0` | Simple reaching task leveraging a via point, which supports self collision detection. Provides a reward only at 100 and 199 for reaching the viapoint and goal point, respectively. | 200 | 5 | 18 |
| `fancy/HoleReacher-v0` | 5 link reaching task where the end-effector needs to reach into a narrow hole without collding with itself or walls. | 200 | 5 | 18 |
## MP Environments
| Name | Description | Horizon | Action Dimension | Context Dimension |
| ----------------------------------- | -------------------------------------------------------------------------------------------------------- | ------- | ---------------- | ----------------- |
| `fancy_DMP/ViaPointReacher-v0` | A DMP provides a trajectory for the `fancy/ViaPointReacher-v0` task. | 200 | 25 |
| `fancy_DMP/HoleReacherFixedGoal-v0` | A DMP provides a trajectory for the `fancy/HoleReacher-v0` task with a fixed goal attractor. | 200 | 25 |
| `fancy_DMP/HoleReacher-v0` | A DMP provides a trajectory for the `fancy/HoleReacher-v0` task. The goal attractor needs to be learned. | 200 | 30 |
[//]: |`fancy/HoleReacherProMPP-v0`|
docs/build/html/_sources/envs/fancy/index.rst.txt
Vendored
+17
View File
@@ -0,0 +1,17 @@
Fancy
=====
Fancy Gym adds a couple new environments. Some of these are adaptations of existing environments while others were designed and build by us from the ground up.
.. raw:: html
<div class='center'>
<img src="../../_static/imgs/env_gifs/Box_Pushing.gif" style="margin: 5%; width: 45%;">
<p>Box Pushing Task (<a href="./mujoco.html#box-pushing">fancy/BoxPushingDense-v0</a>)</p>
</div>
<br>
.. toctree::
mujoco.md
airhockey
classic_control.md
docs/build/html/_sources/envs/fancy/mujoco.md.txt
Vendored
+121
View File
@@ -0,0 +1,121 @@
# Mujoco
## Step-based Environments
### Box Pushing
<div class='center'>
<img src="../../_static/imgs/env_gifs/Box_Pushing.gif" style="margin: 5%; width: 45%;">
</div>
The box-pushing task presents an advanced environment for reinforcement learning (RL) systems, utilizing the versatile Franka Emika Panda robotic arm, which boasts seven degrees of freedom (DoFs). The objective of this task is to precisely manipulate a box to a specified goal location and orientation.
This environment defines its context space with a goal position constrained within a certain range along the x and y axes and a goal orientation that encompasses the full 360-degree range on the z-axis. The robot's mission is to achieve positional accuracy within 5 centimeters and an orientation accuracy within 0.5 radians of the specified goal.
The observation space includes the sine and cosine values of the robotic joint angles, their velocities, and quaternion orientations for the end-effector and the box. The action space describes the applied torques for each joint.
A composite reward function serves as the performance metric for the RL system. It accounts for the distance to the goal, the box's orientation, maintaining a rod within the box, achieving the rod's desired orientation, and includes penalties for joint position and velocity limit violations, as well as an action cost for energy expenditure.
Variations of this environment are available, differing in reward structures and the optionality of randomizing the box's initial position. These variations are purposefully designed to challenge RL algorithms, enhancing their generalization and adaptation capabilities. Temporally sparse environments only provide a reward at the last timestep. Spatially sparse environments only provide a reward, if the goal is almost reached, the box is close enought to the goal and somewhat correctly aligned.
| Name | Description | Horizon | Action Dimension | Observation Dimension |
| ------------------------------------------ | -------------------------------------------------------------------- | ------- | ---------------- | --------------------- |
| `fancy/BoxPushingDense-v0` | Custom Box-pushing task with dense rewards | 100 | 3 | 13 |
| `fancy/BoxPushingTemporalSparse-v0` | Custom Box-pushing task with temporally sparse rewards | 100 | 3 | 13 |
| `fancy/BoxPushingTemporalSpatialSparse-v0` | Custom Box-pushing task with temporally and spatially sparse rewards | 100 | 3 | 13 |
---
### Table Tennis
<div class='center'>
<img src="../../_static/imgs/env_gifs/Table_Tennis.gif" style="margin: 5%; width: 45%;">
</div>
The table tennis task offers a dynamic and interactive environment designed for the development and testing of reinforcement learning (RL) systems. Using a robotic arm equipped with seven degrees of freedom (DoFs), the challenge is to respond to incoming balls and return them accurately to a specified goal location on the opponent's side of the table.
The context space for this environment includes the initial ball position, with x-coordinates ranging from -1 to -0.2 meters and y-coordinates from -0.65 to 0.65 meters, and the goal position with x-coordinates between -1.2 to -0.2 meters and y-coordinates from -0.6 to 0.6 meters. The full observation space comprises the sine and cosine values of the joint angles, the joint velocities, and the ball's velocity, providing comprehensive information for the RL system to base its decisions on.
A task is considered successfully completed when the returned ball not only lands on the opponent's side of the table but also within a tight margin of 20 centimeters from the goal location. The reward function is designed to reflect various conditions of play, including whether the ball was hit, if it landed on the table, and the proximity of the ball's landing position to the goal location.
Variations of the table tennis environment are available to cater to different research needs. These variations maintain the foundational challenge of precise ball return while providing additional complexity for RL algorithms to overcome.
| Name | Description | Horizon | Action Dimension | Observation Dimension |
| ----------------------------------- | -------------------------------------------------------------------------------------------------- | ------- | ---------------- | --------------------- |
| `fancy/TableTennis2D-v0` | Table Tennis task with 2D context, based on a custom environment for table tennis | 350 | 7 | 19 |
| `fancy/TableTennis2DReplan-v0` | Table Tennis task with 2D context and replanning, based on a custom environment for table tennis | 350 | 7 | 19 |
| `fancy/TableTennis4D-v0` | Table Tennis task with 4D context, based on a custom environment for table tennis | 350 | 7 | 22 |
| `fancy/TableTennis4DReplan-v0` | Table Tennis task with 4D context and replanning, based on a custom environment for table tennis | 350 | 7 | 22 |
| `fancy/TableTennisWind-v0` | Table Tennis task with wind effects, based on a custom environment for table tennis | 350 | 7 | 19 |
| `fancy/TableTennisGoalSwitching-v0` | Table Tennis task with goal switching, based on a custom environment for table tennis | 350 | 7 | 19 |
| `fancy/TableTennisWindReplan-v0` | Table Tennis task with wind effects and replanning, based on a custom environment for table tennis | 350 | 7 | 19 |
---
### Beer Pong
<div class='center'>
<img src="../../_static/imgs/env_gifs/Beer_Pong.gif" style="margin: 5%; width: 45%;">
</div>
<!-- TODO: Vid is ugly and unsuccessful. Replace. -->
The Beer Pong task is based upon a robotic system with seven Degrees of Freedom (DoF), challenging the robot to throw a ball into a cup placed on a large table. The environment's context is established by the cup's location, defined within a range of x-coordinates from -1.42 to 1.42 meters and y-coordinates from -4.05 to -1.25 meters.
The observation space includes the cosine and sine of the robot's joint angles, the angular velocities, and distances of the ball relative to the top and bottom of the cup, along with the cup's position and the current timestep. The action space for the robot is defined by the torques applied to each joint. For episode-based methods, the parameter space is expanded to 15 dimensions, which includes two weights for the basis functions per joint and the duration of the throw, namely the ball release time.
Action penalties are implemented in the form of squared torque sums applied across all joints, penalizing excessive force and encouraging efficient motion. The reward function at each timestep t before the final timestep T penalizes the action penalty, while at t=T, a non-Markovian reward based on the ball's position relative to the cup and the action penalty is considered.
Conditions for the task are specified as follows:
- The ball contacts the ground before touching the table.
- The ball is not in the cup and has not made contact with the table.
- The ball is not in the cup but has made contact with the table.
- The ball successfully lands in the cup.
An additional reward component at the final timestep T assesses the chosen ball release time to ensure it falls within a reasonable range. The overall return for an episode is the sum of the rewards at each timestep, the task-specific reward, and the release time reward.
A successful throw in this task is determined by the ball landing in the cup at the episode's conclusion, showcasing the robot's ability to accurately predict and execute the complex motion required for this popular party game.
| Name | Description | Horizon | Action Dimension | Observation Dimension |
| ------------------------------- | ---------------------------------------------------------------------------------------------- | ------- | ---------------- | --------------------- |
| `fancy/BeerPong-v0` | Beer Pong task, based on a custom environment with multiple task variations | 300 | 3 | 29 |
| `fancy/BeerPongStepBased-v0` | Step-based rewards for the Beer Pong task, based on a custom environment with episodic rewards | 300 | 3 | 29 |
| `fancy/BeerPongFixedRelease-v0` | Beer Pong with fixed release, based on a custom environment with episodic rewards | 300 | 3 | 29 |
---
### Variations of existing environments
| Name | Description | Horizon | Action Dimension | Observation Dimension |
| ------------------------------ | ------------------------------------------------------------------------------------------------ | ------- | ---------------- | --------------------- |
| `fancy/Reacher-v0` | Modified (5 links) gymnasiums's mujoco `Reacher-v2` (2 links) | 200 | 5 | 21 |
| `fancy/ReacherSparse-v0` | Same as `fancy/Reacher-v0`, but the distance penalty is only provided in the last time step. | 200 | 5 | 21 |
| `fancy/LongReacher-v0` | Modified (7 links) gymnasiums's mujoco `Reacher-v2` (2 links) | 200 | 7 | 27 |
| `fancy/LongReacherSparse-v0` | Same as `fancy/LongReacher-v0`, but the distance penalty is only provided in the last time step. | 200 | 7 | 27 |
| `fancy/Reacher5d-v0` | Reacher task with 5 links, based on Gymnasium's `gym.envs.mujoco.ReacherEnv` | 200 | 5 | 20 |
| `fancy/Reacher5dSparse-v0` | Sparse Reacher task with 5 links, based on Gymnasium's `gym.envs.mujoco.ReacherEnv` | 200 | 5 | 20 |
| `fancy/Reacher7d-v0` | Reacher task with 7 links, based on Gymnasium's `gym.envs.mujoco.ReacherEnv` | 200 | 7 | 22 |
| `fancy/Reacher7dSparse-v0` | Sparse Reacher task with 7 links, based on Gymnasium's `gym.envs.mujoco.ReacherEnv` | 200 | 7 | 22 |
| `fancy/HopperJumpSparse-v0` | Hopper Jump task with sparse rewards, based on Gymnasium's `gym.envs.mujoco.Hopper` | 250 | 3 | 15 / 16\* |
| `fancy/HopperJump-v0` | Hopper Jump task with continuous rewards, based on Gymnasium's `gym.envs.mujoco.Hopper` | 250 | 3 | 15 / 16\* |
| `fancy/AntJump-v0` | Ant Jump task, based on Gymnasium's `gym.envs.mujoco.Ant` | 200 | 8 | 119 |
| `fancy/HalfCheetahJump-v0` | HalfCheetah Jump task, based on Gymnasium's `gym.envs.mujoco.HalfCheetah` | 100 | 6 | 112 |
| `fancy/HopperJumpOnBox-v0` | Hopper Jump on Box task, based on Gymnasium's `gym.envs.mujoco.Hopper` | 250 | 4 | 16 / 100\* |
| `fancy/HopperThrow-v0` | Hopper Throw task, based on Gymnasium's `gym.envs.mujoco.Hopper` | 250 | 3 | 18 / 100\* |
| `fancy/HopperThrowInBasket-v0` | Hopper Throw in Basket task, based on Gymnasium's `gym.envs.mujoco.Hopper` | 250 | 3 | 18 / 100\* |
| `fancy/Walker2DJump-v0` | Walker 2D Jump task, based on Gymnasium's `gym.envs.mujoco.Walker2d` | 300 | 6 | 18 / 19\* |
\*Observation dimensions depend on configuration.
<!--
No longer used?
| Name | Description | Horizon | Action Dimension | Observation Dimension |
| --------------------------- | --------------------------------------------------------------------------------------------------- | ------- | ---------------- | --------------------- |
| `fancy/BallInACupSimple-v0` | Ball-in-a-cup task where a robot needs to catch a ball attached to a cup at its end-effector. | 4000 | 3 | wip |
| `fancy/BallInACup-v0` | Ball-in-a-cup task where a robot needs to catch a ball attached to a cup at its end-effector | 4000 | 7 | wip |
| `fancy/BallInACupGoal-v0` | Similar to `fancy/BallInACupSimple-v0` but the ball needs to be caught at a specified goal position | 4000 | 7 | wip |
-->
## MP Environments
Many of these envs also exist as MP-variants. Refer to them using `fancy_DMP/<name>` `fancy_ProMP/<name>` or `fancy_ProDMP/<name>`.
docs/build/html/_sources/envs/meta.md.txt
Vendored
+70
View File
@@ -0,0 +1,70 @@
# Metaworld
<div class='center'>
<img src="../_static/imgs/env_gifs/Metaworld.gif" style="margin: 5%; width: 45%;">
<p>Metaworld Dial Turn Task (metaworld/dial-turn-v2)</p>
</div>
<br>
[Metaworld](https://meta-world.github.io/) is an open-source simulated benchmark designed to advance meta-reinforcement learning and multi-task learning, comprising 50 diverse robotic manipulation tasks. The benchmark features a universal tabletop environment equipped with a simulated Sawyer arm and a variety of everyday objects. This shared environment is pivotal for reusing structured learning and efficiently acquiring related tasks.
## Step-Based Environments
`fancy_gym` makes all metaworld ML1 tasks avaible via the standard gym interface.
| Name | Description | Horizon | Action Dimension | Observation Dimension | Context Dimension |
| ---------------------------------------- | ------------------------------------------------------------------------------------- | ------- | ---------------- | --------------------- | ----------------- |
| `metaworld/assembly-v2` | A task where the robot must assemble components. | 150 | 4 | 39 | 6 |
| `metaworld/basketball-v2` | A task where the robot must play a game of basketball. | 150 | 4 | 39 | 6 |
| `metaworld/bin-picking-v2` | A task involving the robot picking objects from a bin. | 150 | 4 | 39 | 6 |
| `metaworld/box-close-v2` | A task requiring the robot to close a box. | 150 | 4 | 39 | 6 |
| `metaworld/button-press-topdown-v2` | A task where the robot must press a button from a top-down perspective. | 150 | 4 | 39 | 6 |
| `metaworld/button-press-topdown-wall-v2` | A task involving the robot pressing a button with a wall from a top-down perspective. | 150 | 4 | 39 | 6 |
| `metaworld/button-press-v2` | A task where the robot must press a button. | 150 | 4 | 39 | 6 |
| `metaworld/button-press-wall-v2` | A task involving the robot pressing a button with a wall. | 150 | 4 | 39 | 6 |
| `metaworld/coffee-button-v2` | A task where the robot must press a button on a coffee machine. | 150 | 4 | 39 | 6 |
| `metaworld/coffee-pull-v2` | A task involving the robot pulling a lever on a coffee machine. | 150 | 4 | 39 | 6 |
| `metaworld/coffee-push-v2` | A task involving the robot pushing a component on a coffee machine. | 150 | 4 | 39 | 6 |
| `metaworld/dial-turn-v2` | A task where the robot must turn a dial. | 150 | 4 | 39 | 6 |
| `metaworld/disassemble-v2` | A task requiring the robot to disassemble an object. | 150 | 4 | 39 | 6 |
| `metaworld/door-close-v2` | A task where the robot must close a door. | 150 | 4 | 39 | 6 |
| `metaworld/door-lock-v2` | A task involving the robot locking a door. | 150 | 4 | 39 | 6 |
| `metaworld/door-open-v2` | A task where the robot must open a door. | 150 | 4 | 39 | 6 |
| `metaworld/door-unlock-v2` | A task involving the robot unlocking a door. | 150 | 4 | 39 | 6 |
| `metaworld/hand-insert-v2` | A task requiring the robot to insert a hand into an object. | 150 | 4 | 39 | 6 |
| `metaworld/drawer-close-v2` | A task where the robot must close a drawer. | 150 | 4 | 39 | 6 |
| `metaworld/drawer-open-v2` | A task involving the robot opening a drawer. | 150 | 4 | 39 | 6 |
| `metaworld/faucet-open-v2` | A task requiring the robot to open a faucet. | 150 | 4 | 39 | 6 |
| `metaworld/faucet-close-v2` | A task where the robot must close a faucet. | 150 | 4 | 39 | 6 |
| `metaworld/hammer-v2` | A task where the robot must use a hammer. | 150 | 4 | 39 | 6 |
| `metaworld/handle-press-side-v2` | A task involving the robot pressing a handle from the side. | 150 | 4 | 39 | 6 |
| `metaworld/handle-press-v2` | A task where the robot must press a handle. | 150 | 4 | 39 | 6 |
| `metaworld/handle-pull-side-v2` | A task requiring the robot to pull a handle from the side. | 150 | 4 | 39 | 6 |
| `metaworld/handle-pull-v2` | A task where the robot must pull a handle. | 150 | 4 | 39 | 6 |
| `metaworld/lever-pull-v2` | A task involving the robot pulling a lever. | 150 | 4 | 39 | 6 |
| `metaworld/peg-insert-side-v2` | A task requiring the robot to insert a peg from the side. | 150 | 4 | 39 | 6 |
| `metaworld/pick-place-wall-v2` | A task involving the robot picking and placing an object with a wall. | 150 | 4 | 39 | 6 |
| `metaworld/pick-out-of-hole-v2` | A task where the robot must pick an object out of a hole. | 150 | 4 | 39 | 6 |
| `metaworld/reach-v2` | A task where the robot must reach an object. | 150 | 4 | 39 | 6 |
| `metaworld/push-back-v2` | A task involving the robot pushing an object backward. | 150 | 4 | 39 | 6 |
| `metaworld/push-v2` | A task where the robot must push an object. | 150 | 4 | 39 | 6 |
| `metaworld/pick-place-v2` | A task involving the robot picking up and placing an object. | 150 | 4 | 39 | 6 |
| `metaworld/plate-slide-v2` | A task requiring the robot to slide a plate. | 150 | 4 | 39 | 6 |
| `metaworld/plate-slide-side-v2` | A task involving the robot sliding a plate from the side. | 150 | 4 | 39 | 6 |
| `metaworld/plate-slide-back-v2` | A task where the robot must slide a plate backward. | 150 | 4 | 39 | 6 |
| `metaworld/plate-slide-back-side-v2` | A task involving the robot sliding a plate backward from the side. | 150 | 4 | 39 | 6 |
| `metaworld/peg-unplug-side-v2` | A task where the robot must unplug a peg from the side. | 150 | 4 | 39 | 6 |
| `metaworld/soccer-v2` | A task where the robot must play soccer. | 150 | 4 | 39 | 6 |
| `metaworld/stick-push-v2` | A task involving the robot pushing a stick. | 150 | 4 | 39 | 6 |
| `metaworld/stick-pull-v2` | A task where the robot must pull a stick. | 150 | 4 | 39 | 6 |
| `metaworld/push-wall-v2` | A task involving the robot pushing against a wall. | 150 | 4 | 39 | 6 |
| `metaworld/reach-wall-v2` | A task where the robot must reach an object with a wall. | 150 | 4 | 39 | 6 |
| `metaworld/shelf-place-v2` | A task involving the robot placing an object on a shelf. | 150 | 4 | 39 | 6 |
| `metaworld/sweep-into-v2` | A task where the robot must sweep objects into a container. | 150 | 4 | 39 | 6 |
| `metaworld/sweep-v2` | A task requiring the robot to sweep. | 150 | 4 | 39 | 6 |
| `metaworld/window-open-v2` | A task where the robot must open a window. | 150 | 4 | 39 | 6 |
| `metaworld/window-close-v2` | A task involving the robot closing a window. | 150 | 4 | 39 | 6 |
## MP Environments
All envs also exist in MP-variants. Refer to them using `metaworld_ProMP/<name-v2>` or `metaworld_ProDMP/<name-v2>` (DMP is currently not supported as of now).
docs/build/html/_sources/envs/open_ai.md.txt
Vendored
+24
View File
@@ -0,0 +1,24 @@
# Gymnasium
<div class='center'>
<img src="../_static/imgs/env_gifs/Lunar_Lander.gif" style="margin: 5%; width: 60%;">
<p>Lunar Lander Task (LunarLander-v2)</p>
</div>
<br>
We provide MP versions for selected [Farama Gymnasium](https://gymnasium.farama.org/) (previously OpenAI Gym) environments.
## Step-Based Environments
We refer to the [Gymnasium docs](https://gymnasium.farama.org/content/basic_usage/) for an overview of step-based environments provided by them.
## MP Environments
These environments are wrapped-versions of their Gymnasium counterparts.
| Name | Description | Trajectory Horizon | Action Dimension |
| ------------------------------------ | -------------------------------------------------------------------- | ------------------ | ---------------- |
| `gym_ProMP/ContinuousMountainCar-v0` | A ProMP wrapped version of the ContinuousMountainCar-v0 environment. | 100 | 1 |
| `gym_ProMP/Reacher-v2` | A ProMP wrapped version of the Reacher-v2 environment. | 50 | 2 |
| `gym_ProMP/FetchSlideDense-v1` | A ProMP wrapped version of the FetchSlideDense-v1 environment. | 50 | 4 |
| `gym_ProMP/FetchReachDense-v1` | A ProMP wrapped version of the FetchReachDense-v1 environment. | 50 | 4 |
docs/build/html/_sources/examples/dmc.rst.txt
Vendored
+6
View File
@@ -0,0 +1,6 @@
DeepMind Control Examples
=========================
.. literalinclude:: ../../../fancy_gym/examples/examples_dmc.py
:language: python
:linenos:
docs/build/html/_sources/examples/general.rst.txt
Vendored
+8
View File
@@ -0,0 +1,8 @@
.. _example-general:
General Usage Examples
======================
.. literalinclude:: ../../../fancy_gym/examples/examples_general.py
:language: python
:linenos:
docs/build/html/_sources/examples/metaworld.rst.txt
Vendored
+6
View File
@@ -0,0 +1,6 @@
Metaworld Examples
==================
.. literalinclude:: ../../../fancy_gym/examples/examples_metaworld.py
:language: python
:linenos:
docs/build/html/_sources/examples/movement_primitives.rst.txt
Vendored
+9
View File
@@ -0,0 +1,9 @@
.. _example-mp:
Movement Primitives Examples
============================
.. literalinclude:: ../../../fancy_gym/examples/examples_movement_primitives.py
:language: python
:linenos:
docs/build/html/_sources/examples/mp_params_tuning.rst.txt
Vendored
+6
View File
@@ -0,0 +1,6 @@
MP Params Tuning Example
========================
.. literalinclude:: ../../../fancy_gym/examples/mp_params_tuning.py
:language: python
:linenos:
docs/build/html/_sources/examples/open_ai.rst.txt
Vendored
+6
View File
@@ -0,0 +1,6 @@
OpenAI Envs Examples
====================
.. literalinclude:: ../../../fancy_gym/examples/examples_open_ai.py
:language: python
:linenos:
docs/build/html/_sources/examples/pd_control_gain_tuning.rst.txt
Vendored
+6
View File
@@ -0,0 +1,6 @@
PD Control Gain Tuning Example
==============================
.. literalinclude:: ../../../fancy_gym/examples/pd_control_gain_tuning.py
:language: python
:linenos:
docs/build/html/_sources/examples/replanning_envs.rst.txt
Vendored
+6
View File
@@ -0,0 +1,6 @@
Replanning Example
==================
.. literalinclude:: ../../../fancy_gym/examples/example_replanning_envs.py
:language: python
:linenos:
docs/build/html/_sources/generated/fancy_gym.envs.rst.txt
Vendored
+23
View File
@@ -0,0 +1,23 @@
fancy\_gym.envs
===============
.. automodule:: fancy_gym.envs
docs/build/html/_sources/generated/fancy_gym.register.rst.txt
Vendored
+6
View File
@@ -0,0 +1,6 @@
fancy\_gym.register
===================
.. currentmodule:: fancy_gym
.. autofunction:: register
docs/build/html/_sources/generated/fancy_gym.upgrade.rst.txt
Vendored
+6
View File
@@ -0,0 +1,6 @@
fancy\_gym.upgrade
==================
.. currentmodule:: fancy_gym
.. autofunction:: upgrade
docs/build/html/_sources/guide/basic_usage.rst.txt
Vendored
+125
View File
@@ -0,0 +1,125 @@
Basic Usage
-----------
We will only show the basics here and prepared :ref:`multiple examples <example-general>` for a more detailed look.
Step-Based Environments
~~~~~~~~~~~~~~~~~~~~~~~
Regular step based environments added by Fancy Gym are added into the
``fancy/`` namespace.
.. note::
Legacy versions of Fancy Gym used ``fancy_gym.make(...)``. This is no longer supported and will raise an Exception on new versions.
.. code:: python
import gymnasium as gym
import fancy_gym
env = gym.make('fancy/Reacher5d-v0')
# or env = gym.make('metaworld/reach-v2') # fancy_gym allows access to all metaworld ML1 tasks via the metaworld/ NS
# or env = gym.make('dm_control/ball_in_cup-catch-v0')
# or env = gym.make('Reacher-v2')
observation = env.reset(seed=1)
for i in range(1000):
action = env.action_space.sample()
observation, reward, terminated, truncated, info = env.step(action)
if i % 5 == 0:
env.render()
if terminated or truncated:
observation, info = env.reset()
Black-Box Environments
~~~~~~~~~~~~~~~~~~~~~~
All environments provide by default the cumulative episode reward, this
can however be changed if necessary. Optionally, each environment
returns all collected information from each step as part of the infos.
This information is, however, mainly meant for debugging as well as
logging and not for training.
+---------------------+--------------------------------------------------------------------------------------------------------------------------------------------+----------+
| Key | Description | Type |
+=====================+============================================================================================================================================+==========+
| `positions` | Generated trajectory from MP | Optional |
+---------------------+--------------------------------------------------------------------------------------------------------------------------------------------+----------+
| `velocities` | Generated trajectory from MP | Optional |
+---------------------+--------------------------------------------------------------------------------------------------------------------------------------------+----------+
| `step_actions` | Step-wise executed action based on controller output | Optional |
+---------------------+--------------------------------------------------------------------------------------------------------------------------------------------+----------+
| `step_observations` | Step-wise intermediate observations | Optional |
+---------------------+--------------------------------------------------------------------------------------------------------------------------------------------+----------+
| `step_rewards` | Step-wise rewards | Optional |
+---------------------+--------------------------------------------------------------------------------------------------------------------------------------------+----------+
| `trajectory_length` | Total number of environment interactions | Always |
+---------------------+--------------------------------------------------------------------------------------------------------------------------------------------+----------+
| `other` | All other information from the underlying environment are returned as a list with length `trajectory_length` maintaining the original key. | Always |
| | In case some information are not provided every time step, the missing values are filled with `None`. | |
+---------------------+--------------------------------------------------------------------------------------------------------------------------------------------+----------+
Existing MP tasks can be created the same way as above. The namespace of
a MP-variant of an environment is given by
``<original namespace>_<MP name>/``. Just keep in mind, calling
``step()`` executes a full trajectory.
.. note::
Currently, we are also in the process of enabling replanning as
well as learning of sub-trajectories. This allows to split the
episode into multiple trajectories and is a hybrid setting between
step-based and black-box leaning. While this is already
implemented, it is still in beta and requires further testing. Feel
free to try it and open an issue with any problems that occur.
.. code:: python
import gymnasium as gym
import fancy_gym
env = gym.make('fancy_ProMP/Reacher5d-v0')
# or env = gym.make('metaworld_ProDMP/reach-v2')
# or env = gym.make('dm_control_DMP/ball_in_cup-catch-v0')
# or env = gym.make('gym_ProMP/Reacher-v2') # mp versions of envs added directly by gymnasium are in the gym_<MP-type> NS
# render() can be called once in the beginning with all necessary arguments.
# To turn it of again just call render() without any arguments.
env.render(mode='human')
# This returns the context information, not the full state observation
observation, info = env.reset(seed=1)
for i in range(5):
action = env.action_space.sample()
observation, reward, terminated, truncated, info = env.step(action)
# terminated or truncated is always True as we are working on the episode level, hence we always reset()
observation, info = env.reset()
To show all available environments, we provide some additional
convenience variables. All of them return a dictionary with the keys
``DMP``, ``ProMP``, ``ProDMP`` and ``all`` that store a list of
available environment ids.
.. code:: python
import fancy_gym
print("All Black-box tasks:")
print(fancy_gym.ALL_MOVEMENT_PRIMITIVE_ENVIRONMENTS)
print("Fancy Black-box tasks:")
print(fancy_gym.ALL_FANCY_MOVEMENT_PRIMITIVE_ENVIRONMENTS)
print("OpenAI Gym Black-box tasks:")
print(fancy_gym.ALL_GYM_MOVEMENT_PRIMITIVE_ENVIRONMENTS)
print("Deepmind Control Black-box tasks:")
print(fancy_gym.ALL_DMC_MOVEMENT_PRIMITIVE_ENVIRONMENTS)
print("MetaWorld Black-box tasks:")
print(fancy_gym.ALL_METAWORLD_MOVEMENT_PRIMITIVE_ENVIRONMENTS)
print("If you add custom envs, their mp versions will be found in:")
print(fancy_gym.MOVEMENT_PRIMITIVE_ENVIRONMENTS_FOR_NS['<my_custom_namespace>'])
docs/build/html/_sources/guide/episodic_rl.rst.txt
Vendored
+36
View File
@@ -0,0 +1,36 @@
What is Episodic RL?
--------------------
.. raw:: html
<div class="justify">
Movement primitive (MP) environments differ from traditional step-based
environments. They align more with concepts from stochastic search,
black-box optimization, and methods commonly found in classical robotics
and control. Instead of individual steps, MP environments operate on an
episode basis, executing complete trajectories. These trajectories are
produced by trajectory generators like Dynamic Movement Primitives
(DMP), Probabilistic Movement Primitives (ProMP) or Probabilistic
Dynamic Movement Primitives (ProDMP).
Once generated, these trajectories are converted into step-by-step
actions using a trajectory tracking controller. The specific controller
chosen depends on the environment’s requirements. Currently, we support
position, velocity, and PD-Controllers tailored for position, velocity,
and torque control. Additionally, we have a specialized controller
designed for the MetaWorld control suite.
While the overarching objective of MP environments remains the learning
of an optimal policy, the actions here represent the parametrization of
motion primitives to craft the right trajectory. Our framework further
enhances this by accommodating a contextual setting. At the episode’s
onset, we present the context space—a subset of the observation space.
This demands the prediction of a new action or MP parametrization for
every unique context.
.. raw:: html
</div>
docs/build/html/_sources/guide/installation.rst.txt
Vendored
+73
View File
@@ -0,0 +1,73 @@
Installation
------------
.. note::
We recommend installing ``fancy_gym`` into a virtual environment as
provided by `venv <https://docs.python.org/3/library/venv.html>`__. 3rd
party alternatives to venv like `Poetry <https://python-poetry.org/>`__
or `Conda <https://docs.conda.io/en/latest/>`__ can also be used.
Installation from PyPI (recommended)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Install ``fancy_gym`` via
.. code:: bash
pip install fancy_gym
We have a few optional dependencies. If you also want to install those
use
.. code:: bash
# to install all optional dependencies
pip install 'fancy_gym[all]'
# or choose only those you want
pip install 'fancy_gym[dmc,box2d,mujoco-legacy,jax,testing]'
Pip can not automatically install up-to-date versions of metaworld,
since they are not avaible on PyPI yet. Install metaworld via
.. code:: bash
pip install metaworld@git+https://github.com/Farama-Foundation/Metaworld.git@d155d0051630bb365ea6a824e02c66c068947439#egg=metaworld
Installation from master
~~~~~~~~~~~~~~~~~~~~~~~~
1. Clone the repository
.. code:: bash
git clone git@github.com:ALRhub/fancy_gym.git
2. Go to the folder
.. code:: bash
cd fancy_gym
3. Install with
.. code:: bash
pip install -e .
We have a few optional dependencies. If you also want to install those
use
.. code:: bash
# to install all optional dependencies
pip install -e '.[all]'
# or choose only those you want
pip install -e '.[dmc,box2d,mujoco-legacy,jax,testing]'
Metaworld has to be installed manually with
.. code:: bash
pip install metaworld@git+https://github.com/Farama-Foundation/Metaworld.git@d155d0051630bb365ea6a824e02c66c068947439#egg=metaworld
docs/build/html/_sources/guide/upgrading_envs.rst.txt
Vendored
+138
View File
@@ -0,0 +1,138 @@
Creating new MP Environments
----------------------------
This guide will explain to you how to upgrade an existing step-based Gymnasium environment into one, that supports Movement Primitives (MPs). If you are looking for a guide to build such a Gymnasium environment instead, please have a look at `this guide <https://gymnasium.farama.org/tutorials/gymnasium_basics/environment_creation/>`__.
In case a required task is not supported yet in the MP framework, it can
be created relatively easy. For the task at hand, the following
`interface <https://github.com/ALRhub/fancy_gym/tree/master/fancy_gym/black_box/raw_interface_wrapper.py>`__
needs to be implemented.
.. code:: python
from abc import abstractmethod
from typing import Union, Tuple
import gymnasium as gym
import numpy as np
class RawInterfaceWrapper(gym.Wrapper):
mp_config = {
'ProMP': {},
'DMP': {},
'ProDMP': {},
}
@property
def context_mask(self) -> np.ndarray:
"""
Returns boolean mask of the same shape as the observation space.
It determines whether the observation is returned for the contextual case or not.
This effectively allows to filter unwanted or unnecessary observations from the full step-based case.
E.g. Velocities starting at 0 are only changing after the first action. Given we only receive the
context/part of the first observation, the velocities are not necessary in the observation for the task.
Returns:
bool array representing the indices of the observations
"""
return np.ones(self.env.observation_space.shape[0], dtype=bool)
@property
@abstractmethod
def current_pos(self) -> Union[float, int, np.ndarray, Tuple]:
"""
Returns the current position of the action/control dimension.
The dimensionality has to match the action/control dimension.
This is not required when exclusively using velocity control,
it should, however, be implemented regardless.
E.g. The joint positions that are directly or indirectly controlled by the action.
"""
raise NotImplementedError()
@property
@abstractmethod
def current_vel(self) -> Union[float, int, np.ndarray, Tuple]:
"""
Returns the current velocity of the action/control dimension.
The dimensionality has to match the action/control dimension.
This is not required when exclusively using position control,
it should, however, be implemented regardless.
E.g. The joint velocities that are directly or indirectly controlled by the action.
"""
raise NotImplementedError()
Default configurations for MPs can be overitten by defining attributes
in mp_config. Available parameters are documented in the `MP_PyTorch
Userguide <https://github.com/ALRhub/MP_PyTorch/blob/main/doc/README.md>`__.
.. code:: python
class RawInterfaceWrapper(gym.Wrapper):
mp_config = {
'ProMP': {
'phase_generator_kwargs': {
'phase_generator_type': 'linear'
# When selecting another generator type, the default configuration will not be merged for the attribute.
},
'controller_kwargs': {
'p_gains': 0.5 * np.array([1.0, 4.0, 2.0, 4.0, 1.0, 4.0, 1.0]),
'd_gains': 0.5 * np.array([0.1, 0.4, 0.2, 0.4, 0.1, 0.4, 0.1]),
},
'basis_generator_kwargs': {
'num_basis': 3,
'num_basis_zero_start': 1,
'num_basis_zero_goal': 1,
},
},
'DMP': {},
'ProDMP': {}.
}
[...]
If you created a new task wrapper, feel free to open a PR, so we can
integrate it for others to use as well. Without the integration the task
can still be used. A rough outline can be shown here, for more details
we recommend having a look at the
:ref:`multiple examples <example-mp>`.
If the step-based is already registered with gym, you can simply do the
following:
.. code:: python
fancy_gym.upgrade(
id='custom/cool_new_env-v0',
mp_wrapper=my_custom_MPWrapper
)
If the step-based is not yet registered with gym we can add both the
step-based and MP-versions via
.. code:: python
fancy_gym.register(
id='custom/cool_new_env-v0',
entry_point=my_custom_env,
mp_wrapper=my_custom_MPWrapper
)
From this point on, you can access MP-version of your environments via
.. code:: python
env = gym.make('custom_ProDMP/cool_new_env-v0')
rewards = 0
observation, info = env.reset()
# number of samples/full trajectories (multiple environment steps)
for i in range(5):
ac = env.action_space.sample()
observation, reward, terminated, truncated, info = env.step(ac)
rewards += reward
if terminated or truncated:
print(rewards)
rewards = 0
observation, info = env.reset()
docs/build/html/_sources/index.rst.txt
Vendored
+166
View File
@@ -0,0 +1,166 @@
Fancy Gym
=========
.. raw:: html
<div style="text-align: center;">
<img src="_static/imgs/fancy_namelogo.svg" style="margin: 5%; width: 80%;"></a>
</div>
<style>
/* Little Hack: We don't want to show the title (ugly), but need to define it since it also sets the pages metadata (for titlebar and stuff) */
h1 {
display: none;
}
</style>
Built upon the foundation of
`Gymnasium <https://gymnasium.farama.org/>`__ (a maintained fork of
OpenAI’s renowned Gym library) ``fancy_gym`` offers a comprehensive
collection of reinforcement learning environments.
Key Features
------------
- **New Challenging Environments**: ``fancy_gym`` includes several new
environments (`Panda Box Pushing <envs/fancy/mujoco.html#box-pushing>`_,
`Table Tennis <envs/fancy/mujoco.html#table-tennis>`_,
`etc. <envs/fancy/index.html>`_) that present a higher degree of
difficulty, pushing the boundaries of reinforcement learning research.
- **Support for Movement Primitives**: ``fancy_gym`` supports a range
of movement primitives (MPs), including Dynamic Movement Primitives
(DMPs), Probabilistic Movement Primitives (ProMP), and Probabilistic
Dynamic Movement Primitives (ProDMP).
- **Upgrade to Movement Primitives**: With our framework, it’s
straightforward to transform standard Gymnasium environments into
environments that support movement primitives.
- **Benchmark Suite Compatibility**: ``fancy_gym`` makes it easy to
access renowned benchmark suites such as `DeepMind
Control <envs/dmc.html>`__
and `Metaworld <envs/meta.html>`__, whether you want
to use them in the regular step-based setting or using MPs.
- **Contribute Your Own Environments**: If you’re inspired to create
custom gym environments, both step-based and with movement
primitives, this
`guide <guide/upgrading_envs.html>`__
will assist you. We encourage and highly appreciate submissions via
PRs to integrate these environments into ``fancy_gym``.
Quickstart Guide
----------------
Install via pip (`or use an alternative installation method <guide/installation.html>`__)
.. code:: bash
pip install 'fancy_gym[all]'
Try out one of our step-based environments (`or explore our other envs <envs/fancy/index.html>`__)
.. code:: python
import gymnasium as gym
import fancy_gym
import time
env = gym.make('fancy/BoxPushingDense-v0', render_mode='human')
observation = env.reset()
env.render()
for i in range(1000):
action = env.action_space.sample() # Randomly sample an action
observation, reward, terminated, truncated, info = env.step(action)
time.sleep(1/env.metadata['render_fps'])
if terminated or truncated:
observation, info = env.reset()
Explore the MP-based variant (`or learn more about Movement Primitives (MPs) <guide/episodic_rl.html>`__)
.. code:: python
import gymnasium as gym
import fancy_gym
env = gym.make('fancy_ProMP/BoxPushingDense-v0', render_mode='human')
env.reset()
env.render()
for i in range(10):
action = env.action_space.sample() # Randomly sample MP parameters
observation, reward, terminated, truncated, info = env.step(action) # Will execute full trajectory, based on MP
observation = env.reset()
.. toctree::
:maxdepth: 3
:caption: User Guide
guide/installation
guide/episodic_rl
guide/basic_usage
guide/upgrading_envs
.. toctree::
:maxdepth: 3
:caption: Environments
envs/fancy/index
envs/dmc
envs/meta
envs/open_ai
.. toctree::
:maxdepth: 3
:caption: Examples
examples/general
examples/dmc
examples/metaworld
examples/open_ai
examples/movement_primitives
examples/mp_params_tuning
examples/pd_control_gain_tuning
examples/replanning_envs
.. toctree::
:maxdepth: 3
:caption: API
api
Citing the Project
------------------
To cite this repository in publications:
.. code:: bibtex
@software{fancy_gym,
title = {Fancy Gym},
author = {Otto, Fabian and Celik, Onur and Roth, Dominik and Zhou, Hongyi},
abstract = {Fancy Gym: Unifying interface for various RL benchmarks with support for Black Box approaches.},
url = {https://github.com/ALRhub/fancy_gym},
organization = {Autonomous Learning Robots Lab (ALR) at KIT},
}
Icon Attribution
----------------
The icon is based on the
`Gymnasium <https://github.com/Farama-Foundation/Gymnasium>`__ icon as
can be found
`here <https://gymnasium.farama.org/_static/img/gymnasium_black.svg>`__.
=================
.. raw:: html
<div style="text-align: center; background: #f8f8f8; border-radius: 10px;">
<a href="https://alr.iar.kit.edu/"><img src="_static/imgs/alr.svg" style="margin: 5%; width: 20%;"></a>
<a href="https://www.kit.edu/"><img src="_static/imgs/kit.svg" style="margin: 5%; width: 20%;"></a>
<a href="https://uni-tuebingen.de/"><img src="_static/imgs/uni_tuebingen.svg" style="margin: 5%; width: 20%;"></a>
</div>
<br>