2024-01-23 17:16:12 +01:00
<!DOCTYPE html>
< html class = "writer-html5" lang = "en" >
< head >
< meta charset = "utf-8" / > < meta name = "generator" content = "Docutils 0.19: https://docutils.sourceforge.io/" / >
< meta name = "viewport" content = "width=device-width, initial-scale=1.0" / >
2024-03-14 15:58:08 +01:00
< title > Movement Primitives Examples — Fancy Gym 0.3.0 documentation< / title >
2024-01-23 17:16:12 +01:00
< link rel = "stylesheet" href = "../_static/pygments.css" type = "text/css" / >
< link rel = "stylesheet" href = "../_static/css/theme.css" type = "text/css" / >
< link rel = "stylesheet" href = "../_static/style.css" type = "text/css" / >
< link rel = "shortcut icon" href = "../_static/icon.svg" / >
<!-- [if lt IE 9]>
< script src = "../_static/js/html5shiv.min.js" > < / script >
<![endif]-->
< script data-url_root = "../" id = "documentation_options" src = "../_static/documentation_options.js" > < / script >
< script src = "../_static/jquery.js" > < / script >
< script src = "../_static/underscore.js" > < / script >
< script src = "../_static/_sphinx_javascript_frameworks_compat.js" > < / script >
< script src = "../_static/doctools.js" > < / script >
< script src = "../_static/sphinx_highlight.js" > < / script >
< script src = "../_static/js/theme.js" > < / script >
< link rel = "index" title = "Index" href = "../genindex.html" / >
< link rel = "search" title = "Search" href = "../search.html" / >
< link rel = "next" title = "MP Params Tuning Example" href = "mp_params_tuning.html" / >
< link rel = "prev" title = "OpenAI Envs Examples" href = "open_ai.html" / >
< / head >
< body class = "wy-body-for-nav" >
< div class = "wy-grid-for-nav" >
< nav data-toggle = "wy-nav-shift" class = "wy-nav-side" >
< div class = "wy-side-scroll" >
< div class = "wy-side-nav-search" >
< a href = "../index.html" class = "icon icon-home" >
Fancy Gym
< img src = "../_static/icon.svg" class = "logo" alt = "Logo" / >
< / a >
< div class = "version" >
2024-03-14 15:58:08 +01:00
0.3.0
2024-01-23 17:16:12 +01:00
< / div >
< div role = "search" >
< form id = "rtd-search-form" class = "wy-form" action = "../search.html" method = "get" >
< input type = "text" name = "q" placeholder = "Search docs" aria-label = "Search docs" / >
< input type = "hidden" name = "check_keywords" value = "yes" / >
< input type = "hidden" name = "area" value = "default" / >
< / form >
< / div >
< / div > < div class = "wy-menu wy-menu-vertical" data-spy = "affix" role = "navigation" aria-label = "Navigation menu" >
< p class = "caption" role = "heading" > < span class = "caption-text" > User Guide< / span > < / p >
< ul >
< li class = "toctree-l1" > < a class = "reference internal" href = "../guide/installation.html" > Installation< / a > < / li >
< li class = "toctree-l1" > < a class = "reference internal" href = "../guide/episodic_rl.html" > What is Episodic RL?< / a > < / li >
< li class = "toctree-l1" > < a class = "reference internal" href = "../guide/basic_usage.html" > Basic Usage< / a > < / li >
< li class = "toctree-l1" > < a class = "reference internal" href = "../guide/upgrading_envs.html" > Creating new MP Environments< / a > < / li >
< / ul >
< p class = "caption" role = "heading" > < span class = "caption-text" > Environments< / span > < / p >
< ul >
< li class = "toctree-l1" > < a class = "reference internal" href = "../envs/fancy/index.html" > Fancy< / a > < / li >
< li class = "toctree-l1" > < a class = "reference internal" href = "../envs/dmc.html" > DeepMind Control (DMC)< / a > < / li >
< li class = "toctree-l1" > < a class = "reference internal" href = "../envs/meta.html" > Metaworld< / a > < / li >
< li class = "toctree-l1" > < a class = "reference internal" href = "../envs/open_ai.html" > Gymnasium< / a > < / li >
< / ul >
< p class = "caption" role = "heading" > < span class = "caption-text" > Examples< / span > < / p >
< ul class = "current" >
< li class = "toctree-l1" > < a class = "reference internal" href = "general.html" > General Usage Examples< / a > < / li >
< li class = "toctree-l1" > < a class = "reference internal" href = "dmc.html" > DeepMind Control Examples< / a > < / li >
< li class = "toctree-l1" > < a class = "reference internal" href = "metaworld.html" > Metaworld Examples< / a > < / li >
< li class = "toctree-l1" > < a class = "reference internal" href = "open_ai.html" > OpenAI Envs Examples< / a > < / li >
< li class = "toctree-l1 current" > < a class = "current reference internal" href = "#" > Movement Primitives Examples< / a > < / li >
< li class = "toctree-l1" > < a class = "reference internal" href = "mp_params_tuning.html" > MP Params Tuning Example< / a > < / li >
< li class = "toctree-l1" > < a class = "reference internal" href = "pd_control_gain_tuning.html" > PD Control Gain Tuning Example< / a > < / li >
< li class = "toctree-l1" > < a class = "reference internal" href = "replanning_envs.html" > Replanning Example< / a > < / li >
< / ul >
< p class = "caption" role = "heading" > < span class = "caption-text" > API< / span > < / p >
< ul >
< li class = "toctree-l1" > < a class = "reference internal" href = "../api.html" > API< / a > < / li >
< / ul >
< / div >
< / div >
< / nav >
< section data-toggle = "wy-nav-shift" class = "wy-nav-content-wrap" > < nav class = "wy-nav-top" aria-label = "Mobile navigation menu" >
< i data-toggle = "wy-nav-top" class = "fa fa-bars" > < / i >
< a href = "../index.html" > Fancy Gym< / a >
< / nav >
< div class = "wy-nav-content" >
< div class = "rst-content" >
< div role = "navigation" aria-label = "Page navigation" >
< ul class = "wy-breadcrumbs" >
< li > < a href = "../index.html" class = "icon icon-home" aria-label = "Home" > < / a > < / li >
< li class = "breadcrumb-item active" > Movement Primitives Examples< / li >
< li class = "wy-breadcrumbs-aside" >
< a href = "https://github.com/ALRhub/fancy_gym/blob/release/docs/source/examples/movement_primitives.rst" class = "fa fa-github" > Edit on GitHub< / a >
< / li >
< / ul >
< hr / >
< / div >
< div role = "main" class = "document" itemscope = "itemscope" itemtype = "http://schema.org/Article" >
< div itemprop = "articleBody" >
< section id = "movement-primitives-examples" >
< span id = "example-mp" > < / span > < h1 > Movement Primitives Examples< a class = "headerlink" href = "#movement-primitives-examples" title = "Permalink to this heading" > < / a > < / h1 >
< div class = "highlight-python notranslate" > < div class = "highlight" > < pre > < span > < / span > < span class = "linenos" > 1< / span > < span class = "kn" > import< / span > < span class = "nn" > gymnasium< / span > < span class = "k" > as< / span > < span class = "nn" > gym< / span >
< span class = "linenos" > 2< / span > < span class = "kn" > import< / span > < span class = "nn" > fancy_gym< / span >
< span class = "linenos" > 3< / span >
< span class = "linenos" > 4< / span >
< span class = "linenos" > 5< / span > < span class = "k" > def< / span > < span class = "nf" > example_mp< / span > < span class = "p" > (< / span > < span class = "n" > env_name< / span > < span class = "o" > =< / span > < span class = "s2" > " fancy_ProMP/HoleReacher-v0" < / span > < span class = "p" > ,< / span > < span class = "n" > seed< / span > < span class = "o" > =< / span > < span class = "mi" > 1< / span > < span class = "p" > ,< / span > < span class = "n" > iterations< / span > < span class = "o" > =< / span > < span class = "mi" > 1< / span > < span class = "p" > ,< / span > < span class = "n" > render< / span > < span class = "o" > =< / span > < span class = "kc" > True< / span > < span class = "p" > ):< / span >
< span class = "linenos" > 6< / span > < span class = "w" > < / span > < span class = "sd" > " " " < / span >
< span class = "linenos" > 7< / span > < span class = "sd" > Example for running a black box based environment, which is already registered< / span >
< span class = "linenos" > 8< / span > < span class = "sd" > Args:< / span >
< span class = "linenos" > 9< / span > < span class = "sd" > env_name: Black box env_id< / span >
< span class = "linenos" > 10< / span > < span class = "sd" > seed: seed for deterministic behaviour< / span >
< span class = "linenos" > 11< / span > < span class = "sd" > iterations: Number of rollout steps to run< / span >
< span class = "linenos" > 12< / span > < span class = "sd" > render: Render the episode< / span >
< span class = "linenos" > 13< / span >
< span class = "linenos" > 14< / span > < span class = "sd" > Returns:< / span >
< span class = "linenos" > 15< / span >
< span class = "linenos" > 16< / span > < span class = "sd" > " " " < / span >
< span class = "linenos" > 17< / span > < span class = "c1" > # Equivalent to gym, we have a make function which can be used to create environments.< / span >
< span class = "linenos" > 18< / span > < span class = "c1" > # It takes care of seeding and enables the use of a variety of external environments using the gym interface.< / span >
< span class = "linenos" > 19< / span > < span class = "n" > env< / span > < span class = "o" > =< / span > < span class = "n" > gym< / span > < span class = "o" > .< / span > < span class = "n" > make< / span > < span class = "p" > (< / span > < span class = "n" > env_name< / span > < span class = "p" > ,< / span > < span class = "n" > render_mode< / span > < span class = "o" > =< / span > < span class = "s1" > ' human' < / span > < span class = "k" > if< / span > < span class = "n" > render< / span > < span class = "k" > else< / span > < span class = "kc" > None< / span > < span class = "p" > )< / span >
< span class = "linenos" > 20< / span >
< span class = "linenos" > 21< / span > < span class = "n" > returns< / span > < span class = "o" > =< / span > < span class = "mi" > 0< / span >
< span class = "linenos" > 22< / span > < span class = "c1" > # env.render(mode=None)< / span >
< span class = "linenos" > 23< / span > < span class = "n" > obs< / span > < span class = "o" > =< / span > < span class = "n" > env< / span > < span class = "o" > .< / span > < span class = "n" > reset< / span > < span class = "p" > (< / span > < span class = "n" > seed< / span > < span class = "o" > =< / span > < span class = "n" > seed< / span > < span class = "p" > )< / span >
< span class = "linenos" > 24< / span >
< span class = "linenos" > 25< / span > < span class = "c1" > # number of samples/full trajectories (multiple environment steps)< / span >
< span class = "linenos" > 26< / span > < span class = "k" > for< / span > < span class = "n" > i< / span > < span class = "ow" > in< / span > < span class = "nb" > range< / span > < span class = "p" > (< / span > < span class = "n" > iterations< / span > < span class = "p" > ):< / span >
< span class = "linenos" > 27< / span >
< span class = "linenos" > 28< / span > < span class = "k" > if< / span > < span class = "n" > render< / span > < span class = "ow" > and< / span > < span class = "n" > i< / span > < span class = "o" > %< / span > < span class = "mi" > 1< / span > < span class = "o" > ==< / span > < span class = "mi" > 0< / span > < span class = "p" > :< / span >
2024-03-14 15:58:08 +01:00
< span class = "linenos" > 29< / span > < span class = "c1" > # This renders the full MP trajectory< / span >
< span class = "linenos" > 30< / span > < span class = "c1" > # It is only required to call render() once in the beginning, which renders every consecutive trajectory.< / span >
< span class = "linenos" > 31< / span > < span class = "n" > env< / span > < span class = "o" > .< / span > < span class = "n" > render< / span > < span class = "p" > ()< / span >
< span class = "linenos" > 32< / span >
< span class = "linenos" > 33< / span > < span class = "c1" > # Now the action space is not the raw action but the parametrization of the trajectory generator,< / span >
< span class = "linenos" > 34< / span > < span class = "c1" > # such as a ProMP< / span >
< span class = "linenos" > 35< / span > < span class = "n" > ac< / span > < span class = "o" > =< / span > < span class = "n" > env< / span > < span class = "o" > .< / span > < span class = "n" > action_space< / span > < span class = "o" > .< / span > < span class = "n" > sample< / span > < span class = "p" > ()< / span >
< span class = "linenos" > 36< / span > < span class = "c1" > # This executes a full trajectory and gives back the context (obs) of the last step in the trajectory, or the< / span >
< span class = "linenos" > 37< / span > < span class = "c1" > # full observation space of the last step, if replanning/sub-trajectory learning is used. The ' reward' is equal< / span >
< span class = "linenos" > 38< / span > < span class = "c1" > # to the return of a trajectory. Default is the sum over the step-wise rewards.< / span >
< span class = "linenos" > 39< / span > < span class = "n" > obs< / span > < span class = "p" > ,< / span > < span class = "n" > reward< / span > < span class = "p" > ,< / span > < span class = "n" > terminated< / span > < span class = "p" > ,< / span > < span class = "n" > truncated< / span > < span class = "p" > ,< / span > < span class = "n" > info< / span > < span class = "o" > =< / span > < span class = "n" > env< / span > < span class = "o" > .< / span > < span class = "n" > step< / span > < span class = "p" > (< / span > < span class = "n" > ac< / span > < span class = "p" > )< / span >
< span class = "linenos" > 40< / span > < span class = "c1" > # Aggregated returns< / span >
< span class = "linenos" > 41< / span > < span class = "n" > returns< / span > < span class = "o" > +=< / span > < span class = "n" > reward< / span >
< span class = "linenos" > 42< / span >
< span class = "linenos" > 43< / span > < span class = "k" > if< / span > < span class = "n" > terminated< / span > < span class = "ow" > or< / span > < span class = "n" > truncated< / span > < span class = "p" > :< / span >
< span class = "linenos" > 44< / span > < span class = "nb" > print< / span > < span class = "p" > (< / span > < span class = "n" > reward< / span > < span class = "p" > )< / span >
< span class = "linenos" > 45< / span > < span class = "n" > obs< / span > < span class = "o" > =< / span > < span class = "n" > env< / span > < span class = "o" > .< / span > < span class = "n" > reset< / span > < span class = "p" > ()< / span >
< span class = "linenos" > 46< / span > < span class = "n" > env< / span > < span class = "o" > .< / span > < span class = "n" > close< / span > < span class = "p" > ()< / span >
< span class = "linenos" > 47< / span >
< span class = "linenos" > 48< / span >
< span class = "linenos" > 49< / span > < span class = "k" > def< / span > < span class = "nf" > example_custom_mp< / span > < span class = "p" > (< / span > < span class = "n" > env_name< / span > < span class = "o" > =< / span > < span class = "s2" > " fancy_ProMP/Reacher5d-v0" < / span > < span class = "p" > ,< / span > < span class = "n" > seed< / span > < span class = "o" > =< / span > < span class = "mi" > 1< / span > < span class = "p" > ,< / span > < span class = "n" > iterations< / span > < span class = "o" > =< / span > < span class = "mi" > 1< / span > < span class = "p" > ,< / span > < span class = "n" > render< / span > < span class = "o" > =< / span > < span class = "kc" > True< / span > < span class = "p" > ):< / span >
< span class = "linenos" > 50< / span > < span class = "w" > < / span > < span class = "sd" > " " " < / span >
< span class = "linenos" > 51< / span > < span class = "sd" > Example for running a custom movement primitive based environments.< / span >
< span class = "linenos" > 52< / span > < span class = "sd" > Our already registered environments follow the same structure.< / span >
< span class = "linenos" > 53< / span > < span class = "sd" > Hence, this also allows to adjust hyperparameters of the movement primitives.< / span >
< span class = "linenos" > 54< / span > < span class = "sd" > Yet, we recommend the method above if you are just interested in changing those parameters for existing tasks.< / span >
< span class = "linenos" > 55< / span > < span class = "sd" > We appreciate PRs for custom environments (especially MP wrappers of existing tasks) < / span >
< span class = "linenos" > 56< / span > < span class = "sd" > for our repo: https://github.com/ALRhub/fancy_gym/< / span >
< span class = "linenos" > 57< / span > < span class = "sd" > Args:< / span >
< span class = "linenos" > 58< / span > < span class = "sd" > seed: seed< / span >
< span class = "linenos" > 59< / span > < span class = "sd" > iterations: Number of rollout steps to run< / span >
< span class = "linenos" > 60< / span > < span class = "sd" > render: Render the episode< / span >
2024-01-23 17:16:12 +01:00
< span class = "linenos" > 61< / span >
2024-03-14 15:58:08 +01:00
< span class = "linenos" > 62< / span > < span class = "sd" > Returns:< / span >
< span class = "linenos" > 63< / span >
< span class = "linenos" > 64< / span > < span class = "sd" > " " " < / span >
< span class = "linenos" > 65< / span > < span class = "c1" > # Changing the arguments of the black box env is possible by providing them to gym through mp_config_override.< / span >
< span class = "linenos" > 66< / span > < span class = "c1" > # E.g. here for way to many basis functions< / span >
< span class = "linenos" > 67< / span > < span class = "n" > env< / span > < span class = "o" > =< / span > < span class = "n" > gym< / span > < span class = "o" > .< / span > < span class = "n" > make< / span > < span class = "p" > (< / span > < span class = "n" > env_name< / span > < span class = "p" > ,< / span > < span class = "n" > seed< / span > < span class = "p" > ,< / span > < span class = "n" > mp_config_override< / span > < span class = "o" > =< / span > < span class = "p" > {< / span > < span class = "s1" > ' basis_generator_kwargs' < / span > < span class = "p" > :< / span > < span class = "p" > {< / span > < span class = "s1" > ' num_basis' < / span > < span class = "p" > :< / span > < span class = "mi" > 1000< / span > < span class = "p" > }},< / span > < span class = "n" > render_mode< / span > < span class = "o" > =< / span > < span class = "s1" > ' human' < / span > < span class = "k" > if< / span > < span class = "n" > render< / span > < span class = "k" > else< / span > < span class = "kc" > None< / span > < span class = "p" > )< / span >
< span class = "linenos" > 68< / span >
< span class = "linenos" > 69< / span > < span class = "n" > returns< / span > < span class = "o" > =< / span > < span class = "mi" > 0< / span >
< span class = "linenos" > 70< / span > < span class = "n" > obs< / span > < span class = "o" > =< / span > < span class = "n" > env< / span > < span class = "o" > .< / span > < span class = "n" > reset< / span > < span class = "p" > ()< / span >
< span class = "linenos" > 71< / span >
< span class = "linenos" > 72< / span > < span class = "c1" > # This time rendering every trajectory< / span >
< span class = "linenos" > 73< / span > < span class = "k" > if< / span > < span class = "n" > render< / span > < span class = "p" > :< / span >
< span class = "linenos" > 74< / span > < span class = "n" > env< / span > < span class = "o" > .< / span > < span class = "n" > render< / span > < span class = "p" > ()< / span >
< span class = "linenos" > 75< / span >
< span class = "linenos" > 76< / span > < span class = "c1" > # number of samples/full trajectories (multiple environment steps)< / span >
< span class = "linenos" > 77< / span > < span class = "k" > for< / span > < span class = "n" > i< / span > < span class = "ow" > in< / span > < span class = "nb" > range< / span > < span class = "p" > (< / span > < span class = "n" > iterations< / span > < span class = "p" > ):< / span >
< span class = "linenos" > 78< / span > < span class = "n" > ac< / span > < span class = "o" > =< / span > < span class = "n" > env< / span > < span class = "o" > .< / span > < span class = "n" > action_space< / span > < span class = "o" > .< / span > < span class = "n" > sample< / span > < span class = "p" > ()< / span >
< span class = "linenos" > 79< / span > < span class = "n" > obs< / span > < span class = "p" > ,< / span > < span class = "n" > reward< / span > < span class = "p" > ,< / span > < span class = "n" > terminated< / span > < span class = "p" > ,< / span > < span class = "n" > truncated< / span > < span class = "p" > ,< / span > < span class = "n" > info< / span > < span class = "o" > =< / span > < span class = "n" > env< / span > < span class = "o" > .< / span > < span class = "n" > step< / span > < span class = "p" > (< / span > < span class = "n" > ac< / span > < span class = "p" > )< / span >
< span class = "linenos" > 80< / span > < span class = "n" > returns< / span > < span class = "o" > +=< / span > < span class = "n" > reward< / span >
< span class = "linenos" > 81< / span >
< span class = "linenos" > 82< / span > < span class = "k" > if< / span > < span class = "n" > terminated< / span > < span class = "ow" > or< / span > < span class = "n" > truncated< / span > < span class = "p" > :< / span >
< span class = "linenos" > 83< / span > < span class = "nb" > print< / span > < span class = "p" > (< / span > < span class = "n" > i< / span > < span class = "p" > ,< / span > < span class = "n" > reward< / span > < span class = "p" > )< / span >
< span class = "linenos" > 84< / span > < span class = "n" > obs< / span > < span class = "o" > =< / span > < span class = "n" > env< / span > < span class = "o" > .< / span > < span class = "n" > reset< / span > < span class = "p" > ()< / span >
< span class = "linenos" > 85< / span >
< span class = "linenos" > 86< / span > < span class = "n" > env< / span > < span class = "o" > .< / span > < span class = "n" > close< / span > < span class = "p" > ()< / span >
< span class = "linenos" > 87< / span > < span class = "k" > return< / span > < span class = "n" > obs< / span >
< span class = "linenos" > 88< / span >
< span class = "linenos" > 89< / span > < span class = "k" > class< / span > < span class = "nc" > Custom_MPWrapper< / span > < span class = "p" > (< / span > < span class = "n" > fancy_gym< / span > < span class = "o" > .< / span > < span class = "n" > envs< / span > < span class = "o" > .< / span > < span class = "n" > mujoco< / span > < span class = "o" > .< / span > < span class = "n" > reacher< / span > < span class = "o" > .< / span > < span class = "n" > MPWrapper< / span > < span class = "p" > ):< / span >
< span class = "linenos" > 90< / span > < span class = "n" > mp_config< / span > < span class = "o" > =< / span > < span class = "p" > {< / span >
< span class = "linenos" > 91< / span > < span class = "s1" > ' ProMP' < / span > < span class = "p" > :< / span > < span class = "p" > {< / span >
< span class = "linenos" > 92< / span > < span class = "s1" > ' trajectory_generator_kwargs' < / span > < span class = "p" > :< / span > < span class = "p" > {< / span >
< span class = "linenos" > 93< / span > < span class = "s1" > ' trajectory_generator_type' < / span > < span class = "p" > :< / span > < span class = "s1" > ' promp' < / span > < span class = "p" > ,< / span >
< span class = "linenos" > 94< / span > < span class = "s1" > ' weights_scale' < / span > < span class = "p" > :< / span > < span class = "mi" > 2< / span >
< span class = "linenos" > 95< / span > < span class = "p" > },< / span >
< span class = "linenos" > 96< / span > < span class = "s1" > ' phase_generator_kwargs' < / span > < span class = "p" > :< / span > < span class = "p" > {< / span >
< span class = "linenos" > 97< / span > < span class = "s1" > ' phase_generator_type' < / span > < span class = "p" > :< / span > < span class = "s1" > ' linear' < / span >
< span class = "linenos" > 98< / span > < span class = "p" > },< / span >
< span class = "linenos" > 99< / span > < span class = "s1" > ' controller_kwargs' < / span > < span class = "p" > :< / span > < span class = "p" > {< / span >
< span class = "linenos" > 100< / span > < span class = "s1" > ' controller_type' < / span > < span class = "p" > :< / span > < span class = "s1" > ' velocity' < / span >
< span class = "linenos" > 101< / span > < span class = "p" > },< / span >
< span class = "linenos" > 102< / span > < span class = "s1" > ' basis_generator_kwargs' < / span > < span class = "p" > :< / span > < span class = "p" > {< / span >
< span class = "linenos" > 103< / span > < span class = "s1" > ' basis_generator_type' < / span > < span class = "p" > :< / span > < span class = "s1" > ' zero_rbf' < / span > < span class = "p" > ,< / span >
< span class = "linenos" > 104< / span > < span class = "s1" > ' num_basis' < / span > < span class = "p" > :< / span > < span class = "mi" > 5< / span > < span class = "p" > ,< / span >
< span class = "linenos" > 105< / span > < span class = "s1" > ' num_basis_zero_start' < / span > < span class = "p" > :< / span > < span class = "mi" > 1< / span >
< span class = "linenos" > 106< / span > < span class = "p" > }< / span >
< span class = "linenos" > 107< / span > < span class = "p" > },< / span >
< span class = "linenos" > 108< / span > < span class = "s1" > ' DMP' < / span > < span class = "p" > :< / span > < span class = "p" > {< / span >
< span class = "linenos" > 109< / span > < span class = "s1" > ' trajectory_generator_kwargs' < / span > < span class = "p" > :< / span > < span class = "p" > {< / span >
< span class = "linenos" > 110< / span > < span class = "s1" > ' trajectory_generator_type' < / span > < span class = "p" > :< / span > < span class = "s1" > ' dmp' < / span > < span class = "p" > ,< / span >
< span class = "linenos" > 111< / span > < span class = "s1" > ' weights_scale' < / span > < span class = "p" > :< / span > < span class = "mi" > 500< / span >
< span class = "linenos" > 112< / span > < span class = "p" > },< / span >
< span class = "linenos" > 113< / span > < span class = "s1" > ' phase_generator_kwargs' < / span > < span class = "p" > :< / span > < span class = "p" > {< / span >
< span class = "linenos" > 114< / span > < span class = "s1" > ' phase_generator_type' < / span > < span class = "p" > :< / span > < span class = "s1" > ' exp' < / span > < span class = "p" > ,< / span >
< span class = "linenos" > 115< / span > < span class = "s1" > ' alpha_phase' < / span > < span class = "p" > :< / span > < span class = "mf" > 2.5< / span >
< span class = "linenos" > 116< / span > < span class = "p" > },< / span >
< span class = "linenos" > 117< / span > < span class = "s1" > ' controller_kwargs' < / span > < span class = "p" > :< / span > < span class = "p" > {< / span >
< span class = "linenos" > 118< / span > < span class = "s1" > ' controller_type' < / span > < span class = "p" > :< / span > < span class = "s1" > ' velocity' < / span >
< span class = "linenos" > 119< / span > < span class = "p" > },< / span >
< span class = "linenos" > 120< / span > < span class = "s1" > ' basis_generator_kwargs' < / span > < span class = "p" > :< / span > < span class = "p" > {< / span >
< span class = "linenos" > 121< / span > < span class = "s1" > ' basis_generator_type' < / span > < span class = "p" > :< / span > < span class = "s1" > ' rbf' < / span > < span class = "p" > ,< / span >
< span class = "linenos" > 122< / span > < span class = "s1" > ' num_basis' < / span > < span class = "p" > :< / span > < span class = "mi" > 5< / span >
< span class = "linenos" > 123< / span > < span class = "p" > }< / span >
< span class = "linenos" > 124< / span > < span class = "p" > }< / span >
< span class = "linenos" > 125< / span > < span class = "p" > }< / span >
< span class = "linenos" > 126< / span >
< span class = "linenos" > 127< / span >
< span class = "linenos" > 128< / span > < span class = "k" > def< / span > < span class = "nf" > example_fully_custom_mp< / span > < span class = "p" > (< / span > < span class = "n" > seed< / span > < span class = "o" > =< / span > < span class = "mi" > 1< / span > < span class = "p" > ,< / span > < span class = "n" > iterations< / span > < span class = "o" > =< / span > < span class = "mi" > 1< / span > < span class = "p" > ,< / span > < span class = "n" > render< / span > < span class = "o" > =< / span > < span class = "kc" > True< / span > < span class = "p" > ):< / span >
< span class = "linenos" > 129< / span > < span class = "w" > < / span > < span class = "sd" > " " " < / span >
< span class = "linenos" > 130< / span > < span class = "sd" > Example for running a custom movement primitive based environments.< / span >
< span class = "linenos" > 131< / span > < span class = "sd" > Our already registered environments follow the same structure.< / span >
< span class = "linenos" > 132< / span > < span class = "sd" > Hence, this also allows to adjust hyperparameters of the movement primitives.< / span >
< span class = "linenos" > 133< / span > < span class = "sd" > Yet, we recommend the method above if you are just interested in changing those parameters for existing tasks.< / span >
< span class = "linenos" > 134< / span > < span class = "sd" > We appreciate PRs for custom environments (especially MP wrappers of existing tasks) < / span >
< span class = "linenos" > 135< / span > < span class = "sd" > for our repo: https://github.com/ALRhub/fancy_gym/< / span >
< span class = "linenos" > 136< / span > < span class = "sd" > Args:< / span >
< span class = "linenos" > 137< / span > < span class = "sd" > seed: seed< / span >
< span class = "linenos" > 138< / span > < span class = "sd" > iterations: Number of rollout steps to run< / span >
< span class = "linenos" > 139< / span > < span class = "sd" > render: Render the episode< / span >
2024-01-23 17:16:12 +01:00
< span class = "linenos" > 140< / span >
2024-03-14 15:58:08 +01:00
< span class = "linenos" > 141< / span > < span class = "sd" > Returns:< / span >
2024-01-23 17:16:12 +01:00
< span class = "linenos" > 142< / span >
2024-03-14 15:58:08 +01:00
< span class = "linenos" > 143< / span > < span class = "sd" > " " " < / span >
< span class = "linenos" > 144< / span >
< span class = "linenos" > 145< / span > < span class = "n" > base_env_id< / span > < span class = "o" > =< / span > < span class = "s2" > " fancy/Reacher5d-v0" < / span >
< span class = "linenos" > 146< / span > < span class = "n" > custom_env_id< / span > < span class = "o" > =< / span > < span class = "s2" > " fancy/Reacher5d-Custom-v0" < / span >
< span class = "linenos" > 147< / span > < span class = "n" > custom_env_id_DMP< / span > < span class = "o" > =< / span > < span class = "s2" > " fancy_DMP/Reacher5d-Custom-v0" < / span >
< span class = "linenos" > 148< / span > < span class = "n" > custom_env_id_ProMP< / span > < span class = "o" > =< / span > < span class = "s2" > " fancy_ProMP/Reacher5d-Custom-v0" < / span >
2024-01-23 17:16:12 +01:00
< span class = "linenos" > 149< / span >
2024-03-14 15:58:08 +01:00
< span class = "linenos" > 150< / span > < span class = "n" > fancy_gym< / span > < span class = "o" > .< / span > < span class = "n" > upgrade< / span > < span class = "p" > (< / span > < span class = "n" > custom_env_id< / span > < span class = "p" > ,< / span > < span class = "n" > mp_wrapper< / span > < span class = "o" > =< / span > < span class = "n" > Custom_MPWrapper< / span > < span class = "p" > ,< / span > < span class = "n" > add_mp_types< / span > < span class = "o" > =< / span > < span class = "p" > [< / span > < span class = "s1" > ' ProMP' < / span > < span class = "p" > ,< / span > < span class = "s1" > ' DMP' < / span > < span class = "p" > ],< / span > < span class = "n" > base_id< / span > < span class = "o" > =< / span > < span class = "n" > base_env_id< / span > < span class = "p" > )< / span >
2024-01-23 17:16:12 +01:00
< span class = "linenos" > 151< / span >
2024-03-14 15:58:08 +01:00
< span class = "linenos" > 152< / span > < span class = "n" > env< / span > < span class = "o" > =< / span > < span class = "n" > gym< / span > < span class = "o" > .< / span > < span class = "n" > make< / span > < span class = "p" > (< / span > < span class = "n" > custom_env_id_ProMP< / span > < span class = "p" > ,< / span > < span class = "n" > render_mode< / span > < span class = "o" > =< / span > < span class = "s1" > ' human' < / span > < span class = "k" > if< / span > < span class = "n" > render< / span > < span class = "k" > else< / span > < span class = "kc" > None< / span > < span class = "p" > )< / span >
< span class = "linenos" > 153< / span >
< span class = "linenos" > 154< / span > < span class = "n" > rewards< / span > < span class = "o" > =< / span > < span class = "mi" > 0< / span >
< span class = "linenos" > 155< / span > < span class = "n" > obs< / span > < span class = "o" > =< / span > < span class = "n" > env< / span > < span class = "o" > .< / span > < span class = "n" > reset< / span > < span class = "p" > ()< / span >
< span class = "linenos" > 156< / span >
< span class = "linenos" > 157< / span > < span class = "k" > if< / span > < span class = "n" > render< / span > < span class = "p" > :< / span >
< span class = "linenos" > 158< / span > < span class = "n" > env< / span > < span class = "o" > .< / span > < span class = "n" > render< / span > < span class = "p" > ()< / span >
< span class = "linenos" > 159< / span >
< span class = "linenos" > 160< / span > < span class = "c1" > # number of samples/full trajectories (multiple environment steps)< / span >
< span class = "linenos" > 161< / span > < span class = "k" > for< / span > < span class = "n" > i< / span > < span class = "ow" > in< / span > < span class = "nb" > range< / span > < span class = "p" > (< / span > < span class = "n" > iterations< / span > < span class = "p" > ):< / span >
< span class = "linenos" > 162< / span > < span class = "n" > ac< / span > < span class = "o" > =< / span > < span class = "n" > env< / span > < span class = "o" > .< / span > < span class = "n" > action_space< / span > < span class = "o" > .< / span > < span class = "n" > sample< / span > < span class = "p" > ()< / span >
< span class = "linenos" > 163< / span > < span class = "n" > obs< / span > < span class = "p" > ,< / span > < span class = "n" > reward< / span > < span class = "p" > ,< / span > < span class = "n" > terminated< / span > < span class = "p" > ,< / span > < span class = "n" > truncated< / span > < span class = "p" > ,< / span > < span class = "n" > info< / span > < span class = "o" > =< / span > < span class = "n" > env< / span > < span class = "o" > .< / span > < span class = "n" > step< / span > < span class = "p" > (< / span > < span class = "n" > ac< / span > < span class = "p" > )< / span >
< span class = "linenos" > 164< / span > < span class = "n" > rewards< / span > < span class = "o" > +=< / span > < span class = "n" > reward< / span >
< span class = "linenos" > 165< / span >
< span class = "linenos" > 166< / span > < span class = "k" > if< / span > < span class = "n" > terminated< / span > < span class = "ow" > or< / span > < span class = "n" > truncated< / span > < span class = "p" > :< / span >
< span class = "linenos" > 167< / span > < span class = "nb" > print< / span > < span class = "p" > (< / span > < span class = "n" > rewards< / span > < span class = "p" > )< / span >
< span class = "linenos" > 168< / span > < span class = "n" > rewards< / span > < span class = "o" > =< / span > < span class = "mi" > 0< / span >
< span class = "linenos" > 169< / span > < span class = "n" > obs< / span > < span class = "o" > =< / span > < span class = "n" > env< / span > < span class = "o" > .< / span > < span class = "n" > reset< / span > < span class = "p" > ()< / span >
< span class = "linenos" > 170< / span >
< span class = "linenos" > 171< / span > < span class = "k" > try< / span > < span class = "p" > :< / span > < span class = "c1" > # Some mujoco-based envs don' t correlcty implement .close< / span >
< span class = "linenos" > 172< / span > < span class = "n" > env< / span > < span class = "o" > .< / span > < span class = "n" > close< / span > < span class = "p" > ()< / span >
< span class = "linenos" > 173< / span > < span class = "k" > except< / span > < span class = "p" > :< / span >
< span class = "linenos" > 174< / span > < span class = "k" > pass< / span >
< span class = "linenos" > 175< / span >
< span class = "linenos" > 176< / span >
< span class = "linenos" > 177< / span > < span class = "k" > def< / span > < span class = "nf" > example_fully_custom_mp_alternative< / span > < span class = "p" > (< / span > < span class = "n" > seed< / span > < span class = "o" > =< / span > < span class = "mi" > 1< / span > < span class = "p" > ,< / span > < span class = "n" > iterations< / span > < span class = "o" > =< / span > < span class = "mi" > 1< / span > < span class = "p" > ,< / span > < span class = "n" > render< / span > < span class = "o" > =< / span > < span class = "kc" > True< / span > < span class = "p" > ):< / span >
< span class = "linenos" > 178< / span > < span class = "w" > < / span > < span class = "sd" > " " " < / span >
< span class = "linenos" > 179< / span > < span class = "sd" > Instead of defining the mp_args in a new custom MP_Wrapper, they can also be provided during registration.< / span >
< span class = "linenos" > 180< / span > < span class = "sd" > Args:< / span >
< span class = "linenos" > 181< / span > < span class = "sd" > seed: seed< / span >
< span class = "linenos" > 182< / span > < span class = "sd" > iterations: Number of rollout steps to run< / span >
< span class = "linenos" > 183< / span > < span class = "sd" > render: Render the episode< / span >
2024-01-23 17:16:12 +01:00
< span class = "linenos" > 184< / span >
2024-03-14 15:58:08 +01:00
< span class = "linenos" > 185< / span > < span class = "sd" > Returns:< / span >
2024-01-23 17:16:12 +01:00
< span class = "linenos" > 186< / span >
2024-03-14 15:58:08 +01:00
< span class = "linenos" > 187< / span > < span class = "sd" > " " " < / span >
< span class = "linenos" > 188< / span >
< span class = "linenos" > 189< / span > < span class = "n" > base_env_id< / span > < span class = "o" > =< / span > < span class = "s2" > " fancy/Reacher5d-v0" < / span >
< span class = "linenos" > 190< / span > < span class = "n" > custom_env_id< / span > < span class = "o" > =< / span > < span class = "s2" > " fancy/Reacher5d-Custom-v0" < / span >
< span class = "linenos" > 191< / span > < span class = "n" > custom_env_id_ProMP< / span > < span class = "o" > =< / span > < span class = "s2" > " fancy_ProMP/Reacher5d-Custom-v0" < / span >
< span class = "linenos" > 192< / span >
< span class = "linenos" > 193< / span > < span class = "n" > fancy_gym< / span > < span class = "o" > .< / span > < span class = "n" > upgrade< / span > < span class = "p" > (< / span > < span class = "n" > custom_env_id< / span > < span class = "p" > ,< / span > < span class = "n" > mp_wrapper< / span > < span class = "o" > =< / span > < span class = "n" > fancy_gym< / span > < span class = "o" > .< / span > < span class = "n" > envs< / span > < span class = "o" > .< / span > < span class = "n" > mujoco< / span > < span class = "o" > .< / span > < span class = "n" > reacher< / span > < span class = "o" > .< / span > < span class = "n" > MPWrapper< / span > < span class = "p" > ,< / span > < span class = "n" > add_mp_types< / span > < span class = "o" > =< / span > < span class = "p" > [< / span > < span class = "s1" > ' ProMP' < / span > < span class = "p" > ],< / span > < span class = "n" > base_id< / span > < span class = "o" > =< / span > < span class = "n" > base_env_id< / span > < span class = "p" > ,< / span > < span class = "n" > mp_config_override< / span > < span class = "o" > =< / span > < span class = "p" > {< / span > < span class = "s1" > ' ProMP' < / span > < span class = "p" > :< / span > < span class = "p" > {< / span >
< span class = "linenos" > 194< / span > < span class = "s1" > ' trajectory_generator_kwargs' < / span > < span class = "p" > :< / span > < span class = "p" > {< / span >
< span class = "linenos" > 195< / span > < span class = "s1" > ' trajectory_generator_type' < / span > < span class = "p" > :< / span > < span class = "s1" > ' promp' < / span > < span class = "p" > ,< / span >
< span class = "linenos" > 196< / span > < span class = "s1" > ' weights_scale' < / span > < span class = "p" > :< / span > < span class = "mi" > 2< / span >
< span class = "linenos" > 197< / span > < span class = "p" > },< / span >
< span class = "linenos" > 198< / span > < span class = "s1" > ' phase_generator_kwargs' < / span > < span class = "p" > :< / span > < span class = "p" > {< / span >
< span class = "linenos" > 199< / span > < span class = "s1" > ' phase_generator_type' < / span > < span class = "p" > :< / span > < span class = "s1" > ' linear' < / span >
< span class = "linenos" > 200< / span > < span class = "p" > },< / span >
< span class = "linenos" > 201< / span > < span class = "s1" > ' controller_kwargs' < / span > < span class = "p" > :< / span > < span class = "p" > {< / span >
< span class = "linenos" > 202< / span > < span class = "s1" > ' controller_type' < / span > < span class = "p" > :< / span > < span class = "s1" > ' velocity' < / span >
< span class = "linenos" > 203< / span > < span class = "p" > },< / span >
< span class = "linenos" > 204< / span > < span class = "s1" > ' basis_generator_kwargs' < / span > < span class = "p" > :< / span > < span class = "p" > {< / span >
< span class = "linenos" > 205< / span > < span class = "s1" > ' basis_generator_type' < / span > < span class = "p" > :< / span > < span class = "s1" > ' zero_rbf' < / span > < span class = "p" > ,< / span >
< span class = "linenos" > 206< / span > < span class = "s1" > ' num_basis' < / span > < span class = "p" > :< / span > < span class = "mi" > 5< / span > < span class = "p" > ,< / span >
< span class = "linenos" > 207< / span > < span class = "s1" > ' num_basis_zero_start' < / span > < span class = "p" > :< / span > < span class = "mi" > 1< / span >
< span class = "linenos" > 208< / span > < span class = "p" > }< / span >
< span class = "linenos" > 209< / span > < span class = "p" > }})< / span >
2024-01-23 17:16:12 +01:00
< span class = "linenos" > 210< / span >
2024-03-14 15:58:08 +01:00
< span class = "linenos" > 211< / span > < span class = "n" > env< / span > < span class = "o" > =< / span > < span class = "n" > gym< / span > < span class = "o" > .< / span > < span class = "n" > make< / span > < span class = "p" > (< / span > < span class = "n" > custom_env_id_ProMP< / span > < span class = "p" > ,< / span > < span class = "n" > render_mode< / span > < span class = "o" > =< / span > < span class = "s1" > ' human' < / span > < span class = "k" > if< / span > < span class = "n" > render< / span > < span class = "k" > else< / span > < span class = "kc" > None< / span > < span class = "p" > )< / span >
< span class = "linenos" > 212< / span >
< span class = "linenos" > 213< / span > < span class = "n" > rewards< / span > < span class = "o" > =< / span > < span class = "mi" > 0< / span >
< span class = "linenos" > 214< / span > < span class = "n" > obs< / span > < span class = "o" > =< / span > < span class = "n" > env< / span > < span class = "o" > .< / span > < span class = "n" > reset< / span > < span class = "p" > ()< / span >
< span class = "linenos" > 215< / span >
< span class = "linenos" > 216< / span > < span class = "k" > if< / span > < span class = "n" > render< / span > < span class = "p" > :< / span >
< span class = "linenos" > 217< / span > < span class = "n" > env< / span > < span class = "o" > .< / span > < span class = "n" > render< / span > < span class = "p" > ()< / span >
< span class = "linenos" > 218< / span >
< span class = "linenos" > 219< / span > < span class = "c1" > # number of samples/full trajectories (multiple environment steps)< / span >
< span class = "linenos" > 220< / span > < span class = "k" > for< / span > < span class = "n" > i< / span > < span class = "ow" > in< / span > < span class = "nb" > range< / span > < span class = "p" > (< / span > < span class = "n" > iterations< / span > < span class = "p" > ):< / span >
< span class = "linenos" > 221< / span > < span class = "n" > ac< / span > < span class = "o" > =< / span > < span class = "n" > env< / span > < span class = "o" > .< / span > < span class = "n" > action_space< / span > < span class = "o" > .< / span > < span class = "n" > sample< / span > < span class = "p" > ()< / span >
< span class = "linenos" > 222< / span > < span class = "n" > obs< / span > < span class = "p" > ,< / span > < span class = "n" > reward< / span > < span class = "p" > ,< / span > < span class = "n" > terminated< / span > < span class = "p" > ,< / span > < span class = "n" > truncated< / span > < span class = "p" > ,< / span > < span class = "n" > info< / span > < span class = "o" > =< / span > < span class = "n" > env< / span > < span class = "o" > .< / span > < span class = "n" > step< / span > < span class = "p" > (< / span > < span class = "n" > ac< / span > < span class = "p" > )< / span >
< span class = "linenos" > 223< / span > < span class = "n" > rewards< / span > < span class = "o" > +=< / span > < span class = "n" > reward< / span >
< span class = "linenos" > 224< / span >
< span class = "linenos" > 225< / span > < span class = "k" > if< / span > < span class = "n" > terminated< / span > < span class = "ow" > or< / span > < span class = "n" > truncated< / span > < span class = "p" > :< / span >
< span class = "linenos" > 226< / span > < span class = "nb" > print< / span > < span class = "p" > (< / span > < span class = "n" > rewards< / span > < span class = "p" > )< / span >
< span class = "linenos" > 227< / span > < span class = "n" > rewards< / span > < span class = "o" > =< / span > < span class = "mi" > 0< / span >
< span class = "linenos" > 228< / span > < span class = "n" > obs< / span > < span class = "o" > =< / span > < span class = "n" > env< / span > < span class = "o" > .< / span > < span class = "n" > reset< / span > < span class = "p" > ()< / span >
< span class = "linenos" > 229< / span >
< span class = "linenos" > 230< / span > < span class = "k" > if< / span > < span class = "n" > render< / span > < span class = "p" > :< / span >
< span class = "linenos" > 231< / span > < span class = "n" > env< / span > < span class = "o" > .< / span > < span class = "n" > render< / span > < span class = "p" > ()< / span >
< span class = "linenos" > 232< / span >
< span class = "linenos" > 233< / span > < span class = "n" > rewards< / span > < span class = "o" > =< / span > < span class = "mi" > 0< / span >
< span class = "linenos" > 234< / span > < span class = "n" > obs< / span > < span class = "o" > =< / span > < span class = "n" > env< / span > < span class = "o" > .< / span > < span class = "n" > reset< / span > < span class = "p" > ()< / span >
< span class = "linenos" > 235< / span >
< span class = "linenos" > 236< / span > < span class = "c1" > # number of samples/full trajectories (multiple environment steps)< / span >
< span class = "linenos" > 237< / span > < span class = "k" > for< / span > < span class = "n" > i< / span > < span class = "ow" > in< / span > < span class = "nb" > range< / span > < span class = "p" > (< / span > < span class = "n" > iterations< / span > < span class = "p" > ):< / span >
< span class = "linenos" > 238< / span > < span class = "n" > ac< / span > < span class = "o" > =< / span > < span class = "n" > env< / span > < span class = "o" > .< / span > < span class = "n" > action_space< / span > < span class = "o" > .< / span > < span class = "n" > sample< / span > < span class = "p" > ()< / span >
< span class = "linenos" > 239< / span > < span class = "n" > obs< / span > < span class = "p" > ,< / span > < span class = "n" > reward< / span > < span class = "p" > ,< / span > < span class = "n" > terminated< / span > < span class = "p" > ,< / span > < span class = "n" > truncated< / span > < span class = "p" > ,< / span > < span class = "n" > info< / span > < span class = "o" > =< / span > < span class = "n" > env< / span > < span class = "o" > .< / span > < span class = "n" > step< / span > < span class = "p" > (< / span > < span class = "n" > ac< / span > < span class = "p" > )< / span >
< span class = "linenos" > 240< / span > < span class = "n" > rewards< / span > < span class = "o" > +=< / span > < span class = "n" > reward< / span >
< span class = "linenos" > 241< / span >
< span class = "linenos" > 242< / span > < span class = "k" > if< / span > < span class = "n" > terminated< / span > < span class = "ow" > or< / span > < span class = "n" > truncated< / span > < span class = "p" > :< / span >
< span class = "linenos" > 243< / span > < span class = "nb" > print< / span > < span class = "p" > (< / span > < span class = "n" > rewards< / span > < span class = "p" > )< / span >
< span class = "linenos" > 244< / span > < span class = "n" > rewards< / span > < span class = "o" > =< / span > < span class = "mi" > 0< / span >
< span class = "linenos" > 245< / span > < span class = "n" > obs< / span > < span class = "o" > =< / span > < span class = "n" > env< / span > < span class = "o" > .< / span > < span class = "n" > reset< / span > < span class = "p" > ()< / span >
< span class = "linenos" > 246< / span >
< span class = "linenos" > 247< / span > < span class = "k" > try< / span > < span class = "p" > :< / span > < span class = "c1" > # Some mujoco-based envs don' t correlcty implement .close< / span >
< span class = "linenos" > 248< / span > < span class = "n" > env< / span > < span class = "o" > .< / span > < span class = "n" > close< / span > < span class = "p" > ()< / span >
< span class = "linenos" > 249< / span > < span class = "k" > except< / span > < span class = "p" > :< / span >
< span class = "linenos" > 250< / span > < span class = "k" > pass< / span >
< span class = "linenos" > 251< / span >
< span class = "linenos" > 252< / span >
< span class = "linenos" > 253< / span > < span class = "k" > def< / span > < span class = "nf" > main< / span > < span class = "p" > (< / span > < span class = "n" > render< / span > < span class = "o" > =< / span > < span class = "kc" > False< / span > < span class = "p" > ):< / span >
< span class = "linenos" > 254< / span > < span class = "c1" > # DMP< / span >
< span class = "linenos" > 255< / span > < span class = "n" > example_mp< / span > < span class = "p" > (< / span > < span class = "s2" > " fancy_DMP/HoleReacher-v0" < / span > < span class = "p" > ,< / span > < span class = "n" > seed< / span > < span class = "o" > =< / span > < span class = "mi" > 10< / span > < span class = "p" > ,< / span > < span class = "n" > iterations< / span > < span class = "o" > =< / span > < span class = "mi" > 5< / span > < span class = "p" > ,< / span > < span class = "n" > render< / span > < span class = "o" > =< / span > < span class = "n" > render< / span > < span class = "p" > )< / span >
< span class = "linenos" > 256< / span >
< span class = "linenos" > 257< / span > < span class = "c1" > # ProMP< / span >
< span class = "linenos" > 258< / span > < span class = "n" > example_mp< / span > < span class = "p" > (< / span > < span class = "s2" > " fancy_ProMP/HoleReacher-v0" < / span > < span class = "p" > ,< / span > < span class = "n" > seed< / span > < span class = "o" > =< / span > < span class = "mi" > 10< / span > < span class = "p" > ,< / span > < span class = "n" > iterations< / span > < span class = "o" > =< / span > < span class = "mi" > 5< / span > < span class = "p" > ,< / span > < span class = "n" > render< / span > < span class = "o" > =< / span > < span class = "n" > render< / span > < span class = "p" > )< / span >
< span class = "linenos" > 259< / span > < span class = "n" > example_mp< / span > < span class = "p" > (< / span > < span class = "s2" > " fancy_ProMP/BoxPushingTemporalSparse-v0" < / span > < span class = "p" > ,< / span > < span class = "n" > seed< / span > < span class = "o" > =< / span > < span class = "mi" > 10< / span > < span class = "p" > ,< / span > < span class = "n" > iterations< / span > < span class = "o" > =< / span > < span class = "mi" > 1< / span > < span class = "p" > ,< / span > < span class = "n" > render< / span > < span class = "o" > =< / span > < span class = "n" > render< / span > < span class = "p" > )< / span >
< span class = "linenos" > 260< / span > < span class = "n" > example_mp< / span > < span class = "p" > (< / span > < span class = "s2" > " fancy_ProMP/TableTennis4D-v0" < / span > < span class = "p" > ,< / span > < span class = "n" > seed< / span > < span class = "o" > =< / span > < span class = "mi" > 10< / span > < span class = "p" > ,< / span > < span class = "n" > iterations< / span > < span class = "o" > =< / span > < span class = "mi" > 20< / span > < span class = "p" > ,< / span > < span class = "n" > render< / span > < span class = "o" > =< / span > < span class = "n" > render< / span > < span class = "p" > )< / span >
< span class = "linenos" > 261< / span >
< span class = "linenos" > 262< / span > < span class = "c1" > # ProDMP with Replanning< / span >
< span class = "linenos" > 263< / span > < span class = "n" > example_mp< / span > < span class = "p" > (< / span > < span class = "s2" > " fancy_ProDMP/BoxPushingDenseReplan-v0" < / span > < span class = "p" > ,< / span > < span class = "n" > seed< / span > < span class = "o" > =< / span > < span class = "mi" > 10< / span > < span class = "p" > ,< / span > < span class = "n" > iterations< / span > < span class = "o" > =< / span > < span class = "mi" > 4< / span > < span class = "p" > ,< / span > < span class = "n" > render< / span > < span class = "o" > =< / span > < span class = "n" > render< / span > < span class = "p" > )< / span >
< span class = "linenos" > 264< / span > < span class = "n" > example_mp< / span > < span class = "p" > (< / span > < span class = "s2" > " fancy_ProDMP/TableTennis4DReplan-v0" < / span > < span class = "p" > ,< / span > < span class = "n" > seed< / span > < span class = "o" > =< / span > < span class = "mi" > 10< / span > < span class = "p" > ,< / span > < span class = "n" > iterations< / span > < span class = "o" > =< / span > < span class = "mi" > 20< / span > < span class = "p" > ,< / span > < span class = "n" > render< / span > < span class = "o" > =< / span > < span class = "n" > render< / span > < span class = "p" > )< / span >
< span class = "linenos" > 265< / span > < span class = "n" > example_mp< / span > < span class = "p" > (< / span > < span class = "s2" > " fancy_ProDMP/TableTennisWindReplan-v0" < / span > < span class = "p" > ,< / span > < span class = "n" > seed< / span > < span class = "o" > =< / span > < span class = "mi" > 10< / span > < span class = "p" > ,< / span > < span class = "n" > iterations< / span > < span class = "o" > =< / span > < span class = "mi" > 20< / span > < span class = "p" > ,< / span > < span class = "n" > render< / span > < span class = "o" > =< / span > < span class = "n" > render< / span > < span class = "p" > )< / span >
< span class = "linenos" > 266< / span >
< span class = "linenos" > 267< / span > < span class = "c1" > # Altered basis functions< / span >
< span class = "linenos" > 268< / span > < span class = "n" > obs1< / span > < span class = "o" > =< / span > < span class = "n" > example_custom_mp< / span > < span class = "p" > (< / span > < span class = "s2" > " fancy_ProMP/Reacher5d-v0" < / span > < span class = "p" > ,< / span > < span class = "n" > seed< / span > < span class = "o" > =< / span > < span class = "mi" > 10< / span > < span class = "p" > ,< / span > < span class = "n" > iterations< / span > < span class = "o" > =< / span > < span class = "mi" > 1< / span > < span class = "p" > ,< / span > < span class = "n" > render< / span > < span class = "o" > =< / span > < span class = "n" > render< / span > < span class = "p" > )< / span >
< span class = "linenos" > 269< / span >
< span class = "linenos" > 270< / span > < span class = "c1" > # Custom MP< / span >
< span class = "linenos" > 271< / span > < span class = "n" > example_fully_custom_mp< / span > < span class = "p" > (< / span > < span class = "n" > seed< / span > < span class = "o" > =< / span > < span class = "mi" > 10< / span > < span class = "p" > ,< / span > < span class = "n" > iterations< / span > < span class = "o" > =< / span > < span class = "mi" > 1< / span > < span class = "p" > ,< / span > < span class = "n" > render< / span > < span class = "o" > =< / span > < span class = "n" > render< / span > < span class = "p" > )< / span >
< span class = "linenos" > 272< / span > < span class = "n" > example_fully_custom_mp_alternative< / span > < span class = "p" > (< / span > < span class = "n" > seed< / span > < span class = "o" > =< / span > < span class = "mi" > 10< / span > < span class = "p" > ,< / span > < span class = "n" > iterations< / span > < span class = "o" > =< / span > < span class = "mi" > 1< / span > < span class = "p" > ,< / span > < span class = "n" > render< / span > < span class = "o" > =< / span > < span class = "n" > render< / span > < span class = "p" > )< / span >
< span class = "linenos" > 273< / span >
< span class = "linenos" > 274< / span > < span class = "k" > if< / span > < span class = "vm" > __name__< / span > < span class = "o" > ==< / span > < span class = "s1" > ' __main__' < / span > < span class = "p" > :< / span >
< span class = "linenos" > 275< / span > < span class = "n" > main< / span > < span class = "p" > ()< / span >
2024-01-23 17:16:12 +01:00
< / pre > < / div >
< / div >
< / section >
< / div >
< / div >
< footer > < div class = "rst-footer-buttons" role = "navigation" aria-label = "Footer" >
< a href = "open_ai.html" class = "btn btn-neutral float-left" title = "OpenAI Envs Examples" accesskey = "p" rel = "prev" > < span class = "fa fa-arrow-circle-left" aria-hidden = "true" > < / span > Previous< / a >
< a href = "mp_params_tuning.html" class = "btn btn-neutral float-right" title = "MP Params Tuning Example" accesskey = "n" rel = "next" > Next < span class = "fa fa-arrow-circle-right" aria-hidden = "true" > < / span > < / a >
< / div >
< hr / >
< div role = "contentinfo" >
< p > © Copyright 2020-2024, Fabian Otto, Onur Celik, Dominik Roth, Hongyi Zhou.< / p >
< / div >
Built with < a href = "https://www.sphinx-doc.org/" > Sphinx< / a > using a
< a href = "https://github.com/readthedocs/sphinx_rtd_theme" > theme< / a >
provided by < a href = "https://readthedocs.org" > Read the Docs< / a > .
< / footer >
< / div >
< / div >
< / section >
< / div >
< script >
jQuery(function () {
SphinxRtdTheme.Navigation.enable(true);
});
< / script >
< / body >
< / html >
