Support for torchrl envs

README.md

@@ -30,8 +30,7 @@ Fancy RL provides two main components:
    model.train()
    ```
 
-   For environments, you can pass any [gymnasium](https://gymnasium.farama.org/) or [Fancy Gym](https://alrhub.github.io/fancy_gym/) environment ID as a string, a function returning a gymnasium environment, or an already instantiated gymnasium environment. Future plans include supporting other torchrl environments.
-   Check 'example/example.py' for a more complete usage example.
+   For environments, you can pass any [gymnasium](https://gymnasium.farama.org/) or [Fancy Gym](https://alrhub.github.io/fancy_gym/) environment ID as a string, a function returning a gymnasium or torchrl environment, an already instantiated gymnasium or torchrl environment, or a dict that will be passed to gymnasium.make. Check 'example/example.py' for a more complete usage example.
 
 2. **Additional Modules for TRPL**: Designed to integrate with torchrl's primitives-first approach, these modules are ideal for building custom algorithms with precise trust region projections.
 

fancy_rl/on_policy.py

@@ -83,6 +83,8 @@ class OnPolicy(ABC):
             env = env_spec()
             if isinstance(env, gym.Env):
                 env = GymWrapper(env)
+        elif isinstance(env, gym.Env):
+            env = GymWrapper(env)
         else:
             raise ValueError("env_spec must be a string or a callable that returns an environment.")
         return env