dodox/dppo
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

update batch size in D3IL so it works with the new form of gradient update

Browse Source
This commit is contained in:
allenzren 2024-12-24 02:06:17 -05:00
parent 1d04211666
commit e7f73dffc1
3 changed files with 3 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
cfg/d3il/finetune/avoid_m1/ft_ppo_diffusion_mlp.yaml
View File

@ -76,7 +76,7 @@ train:
reward_scale_running: True reward_scale_running: True
reward_scale_const: 1.0 reward_scale_const: 1.0
gae_lambda: 0.95 gae_lambda: 0.95
batch_size: ${eval:'round(${train.n_steps} * ${env.n_envs} / 2)'} batch_size: ${eval:'round(${train.n_steps} * ${env.n_envs} * ${ft_denoising_steps} / 2)'}
update_epochs: 10 update_epochs: 10
vf_coef: 0.5 vf_coef: 0.5
target_kl: 1 target_kl: 1

2
cfg/d3il/finetune/avoid_m2/ft_ppo_diffusion_mlp.yaml
View File

@ -76,7 +76,7 @@ train:
reward_scale_running: True reward_scale_running: True
reward_scale_const: 1.0 reward_scale_const: 1.0
gae_lambda: 0.95 gae_lambda: 0.95
batch_size: ${eval:'round(${train.n_steps} * ${env.n_envs} / 2)'} batch_size: ${eval:'round(${train.n_steps} * ${env.n_envs} * ${ft_denoising_steps} / 2)'}
update_epochs: 10 update_epochs: 10
vf_coef: 0.5 vf_coef: 0.5
target_kl: 1 target_kl: 1

2
cfg/d3il/finetune/avoid_m3/ft_ppo_diffusion_mlp.yaml
View File

@ -76,7 +76,7 @@ train:
reward_scale_running: True reward_scale_running: True
reward_scale_const: 1.0 reward_scale_const: 1.0
gae_lambda: 0.95 gae_lambda: 0.95
batch_size: ${eval:'round(${train.n_steps} * ${env.n_envs} / 2)'} batch_size: ${eval:'round(${train.n_steps} * ${env.n_envs} * ${ft_denoising_steps} / 2)'}
update_epochs: 10 update_epochs: 10
vf_coef: 0.5 vf_coef: 0.5
target_kl: 1 target_kl: 1