dodox/dppo
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
59 Commits 1 Branch 0 Tags
main
Commit Graph
4 Commits
Author SHA1 Message Date
allenzren 0bdae945e9 use default epoch_start_ema=20 and update_ema_freq=10 2024-11-07 10:55:16 -05:00
allenzren c0921a1fb5 remove update_ema_freq 2024-11-06 21:01:15 -05:00
Allen Z. Ren e1ef4ca1cf More frequent EMA update (#20) 
* move ema update within pretraining epoch

* update pretraining ema configs

* add lift and can epoch 8000 checkpoint url

* add note about EMA issue in pretraining instruction
 2024-11-06 20:42:31 -05:00
allenzren 8293b0936b release 2024-09-03 21:03:27 -04:00