From 91b10cc90a8841cf60c44a0fa9d05256a5beeab0 Mon Sep 17 00:00:00 2001 From: Dominik Roth Date: Tue, 23 Jan 2024 17:15:24 +0100 Subject: [PATCH] Mimimized README --- README.md | 200 ++++++++++-------------------------------------------- 1 file changed, 36 insertions(+), 164 deletions(-) diff --git a/README.md b/README.md index db662a1..b766c79 100644 --- a/README.md +++ b/README.md @@ -10,192 +10,64 @@ Built upon the foundation of [Gymnasium](https://gymnasium.farama.org/) (a maint **Key Features**: -- **New Challenging Environments**: `fancy_gym` includes several new environments (Panda Box Pushing, Table Tennis, etc.) that present a higher degree of difficulty, pushing the boundaries of reinforcement learning research. +- **New Challenging Environments**: `fancy_gym` includes several new environments ([Panda Box Pushing](https://dominik-roth.eu/fancy/envs/fancy/mujoco.html#box-pushing), [Table Tennis](https://dominik-roth.eu/fancy/envs/fancy/mujoco.html#table-tennis), [etc.](https://dominik-roth.eu/fancy/envs/fancy/index.html)) that present a higher degree of difficulty, pushing the boundaries of reinforcement learning research. - **Support for Movement Primitives**: `fancy_gym` supports a range of movement primitives (MPs), including Dynamic Movement Primitives (DMPs), Probabilistic Movement Primitives (ProMP), and Probabilistic Dynamic Movement Primitives (ProDMP). -- **Upgrade to Movement Primitives**: With our framework, it's straightforward to transform standard Gymnasium environments into environments that support movement primitives. -- **Benchmark Suite Compatibility**: `fancy_gym` makes it easy to access renowned benchmark suites such as [DeepMind Control](https://deepmind.com/research/publications/2020/dm-control-Software-and-Tasks-for-Continuous-Control) and [Metaworld](https://meta-world.github.io/), whether you want to use them in the regular step-based setting or using MPs. -- **Contribute Your Own Environments**: If you're inspired to create custom gym environments, both step-based and with movement primitives, this [guide](https://gymnasium.farama.org/tutorials/gymnasium_basics/environment_creation/) will assist you. We encourage and highly appreciate submissions via PRs to integrate these environments into `fancy_gym`. +- **Upgrade to Movement Primitives**: With our framework, it’s straightforward to transform standard Gymnasium environments into environments that support movement primitives. +- **Benchmark Suite Compatibility**: `fancy_gym` makes it easy to access renowned benchmark suites such as [DeepMind Control](dominik-roth.eu/fancy/envs/dmc.html) + and [Metaworld](https://dominik-roth.eu/fancy/envs/meta.html), whether you want to use them in the regular step-based setting or using MPs. +- **Contribute Your Own Environments**: If you’re inspired to create custom gym environments, both step-based and with movement primitives, this [guide](https://dominik-roth.eu/fancy/guide/upgrading_envs.html) will assist you. We encourage and highly appreciate submissions via PRs to integrate these environments into `fancy_gym`. -## Movement Primitive Environments (Episode-Based/Black-Box Environments) +## Quickstart Guide -

-In step-based environments, actions are determined step by step, with each individual observation directly mapped to a corresponding action. Contrary to this, in episodic MP-based (Movement Primitive-based) environments, the process is different. Here, rather than responding to individual observations, a broader context is considered at the start of each episode. This context is used to define parameters for Movement Primitives (MPs), which then describe a complete trajectory. The trajectory is executed over the entire episode using a tracking controller, allowing for the enactment of complex, continuous sequences of actions. This approach contrasts with the discrete, moment-to-moment decision-making of step-based environments and integrates concepts from stochastic search and black-box optimization, commonly found in classical robotics and control. -

+| ℹ We recommend installing `fancy_gym` into a virtual environment as provided by [venv](https://docs.python.org/3/library/venv.html), [Poetry](https://python-poetry.org/) or [Conda](https://docs.conda.io/en/latest/). +| +| ----------------------------------------------------------------------------------------------------------------------------------------------- | -For a more extensive explaination, please have a look at our Documentation-TODO:Link. - -## Installation - -We recommend installing `fancy_gym` into a virtual environment as provided by [venv](https://docs.python.org/3/library/venv.html). 3rd party alternatives to venv like [Poetry](https://python-poetry.org/) or [Conda](https://docs.conda.io/en/latest/) can also be used. - -### Installation from PyPI (recommended) - -Install `fancy_gym` via +Install via pip (`or use an alternative installation method `\_\_) ```bash -pip install fancy_gym + pip install 'fancy_gym[all]' ``` -We have a few optional dependencies. If you also want to install those use - -```bash -# to install all optional dependencies -pip install 'fancy_gym[all]' - -# or choose only those you want -pip install 'fancy_gym[dmc,box2d,mujoco-legacy,jax,testing]' -``` - -Pip can not automatically install up-to-date versions of metaworld, since they are not avaible on PyPI yet. -Install metaworld via - -```bash -pip install metaworld@git+https://github.com/Farama-Foundation/Metaworld.git@d155d0051630bb365ea6a824e02c66c068947439#egg=metaworld -``` - -### Installation from master - -1. Clone the repository - -```bash -git clone git@github.com:ALRhub/fancy_gym.git -``` - -2. Go to the folder - -```bash -cd fancy_gym -``` - -3. Install with - -```bash -pip install -e . -``` - -We have a few optional dependencies. If you also want to install those use - -```bash -# to install all optional dependencies -pip install -e '.[all]' - -# or choose only those you want -pip install -e '.[dmc,box2d,mujoco-legacy,jax,testing]' -``` - -Metaworld has to be installed manually with - -```bash -pip install metaworld@git+https://github.com/Farama-Foundation/Metaworld.git@d155d0051630bb365ea6a824e02c66c068947439#egg=metaworld -``` - -## How to use Fancy Gym - -Documentation for `fancy_gym` is avaible at TODO:Link. Usage examples can be found here-TODO:Link. - -### Step-Based Environments - -Regular step based environments added by Fancy Gym are added into the `fancy/` namespace. - -| ❗ Legacy versions of Fancy Gym used `fancy_gym.make(...)`. This is no longer supported and will raise an Exception on new versions. | -| ------------------------------------------------------------------------------------------------------------------------------------------ | +Try out one of our step-based environments (`or explore our other envs `\_\_) ```python -import gymnasium as gym -import fancy_gym + import gymnasium as gym + import fancy_gym + import time -env = gym.make('fancy/Reacher5d-v0') -# or env = gym.make('metaworld/reach-v2') # fancy_gym allows access to all metaworld ML1 tasks via the metaworld/ NS -# or env = gym.make('dm_control/ball_in_cup-catch-v0') -# or env = gym.make('Reacher-v2') -observation = env.reset(seed=1) + env = gym.make('fancy/BoxPushingDense-v0', render_mode='human') + observation = env.reset() + env.render() -for i in range(1000): - action = env.action_space.sample() - observation, reward, terminated, truncated, info = env.step(action) - if i % 5 == 0: - env.render() + for i in range(1000): + action = env.action_space.sample() # Randomly sample an action + observation, reward, terminated, truncated, info = env.step(action) + time.sleep(1/env.metadata['render_fps']) - if terminated or truncated: - observation, info = env.reset() + if terminated or truncated: + observation, info = env.reset() ``` -A list of all included environments is avaible here-TODO:Link. - -### Black-box Environments - -Existing MP tasks can be created the same way as above. The namespace of a MP-variant of an environment is given by `_/`. -Just keep in mind, calling `step()` executes a full trajectory. +Explore the MP-based variant (`or learn more about Movement Primitives (MPs) `\_\_) ```python -import gymnasium as gym -import fancy_gym + import gymnasium as gym + import fancy_gym -env = gym.make('fancy_ProMP/Reacher5d-v0') -# or env = gym.make('metaworld_ProDMP/reach-v2') -# or env = gym.make('dm_control_DMP/ball_in_cup-catch-v0') -# or env = gym.make('gym_ProMP/Reacher-v2') # mp versions of envs added directly by gymnasium are in the gym_ NS + env = gym.make('fancy_ProMP/BoxPushingDense-v0', render_mode='human') + env.reset() + env.render() -# render() can be called once in the beginning with all necessary arguments. -# To turn it of again just call render() without any arguments. -env.render(mode='human') - -# This returns the context information, not the full state observation -observation, info = env.reset(seed=1) - -for i in range(5): - action = env.action_space.sample() - observation, reward, terminated, truncated, info = env.step(action) - - # terminated or truncated is always True as we are working on the episode level, hence we always reset() - observation, info = env.reset() + for i in range(10): + action = env.action_space.sample() # Randomly sample MP parameters + observation, reward, terminated, truncated, info = env.step(action) # Will execute full trajectory, based on MP + observation = env.reset() ``` -A list of all included MP environments is avaible here-TODO:Link. +## Documentation -### How to create a new MP task - -We refer to our Documentation for a complete description-TODO:Link. - -If the step-based is already registered with gym, you can simply do the following: - -```python -fancy_gym.upgrade( - id='custom/cool_new_env-v0', - mp_wrapper=my_custom_MPWrapper -) -``` - -If the step-based is not yet registered with gym we can add both the step-based and MP-versions via - -```python -fancy_gym.register( - id='custom/cool_new_env-v0', - entry_point=my_custom_env, - mp_wrapper=my_custom_MPWrapper -) -``` - -As for how to write custom MP-Wrappers, please have a look at our Documentation-TODO:Link. -From this point on, you can access MP-version of your environments via - -```python -env = gym.make('custom_ProDMP/cool_new_env-v0') - -rewards = 0 -observation, info = env.reset() - -# number of samples/full trajectories (multiple environment steps) -for i in range(5): - ac = env.action_space.sample() - observation, reward, terminated, truncated, info = env.step(ac) - rewards += reward - - if terminated or truncated: - print(rewards) - rewards = 0 - observation, info = env.reset() -``` +Documentation for `fancy_gym` can be found [here](https://dominik-roth.eu/fancy); Usage Examples can be found [here](https://dominik-roth.eu/fancy/examples/general.html). ## Citing the Project