4 Commits

Author	SHA1	Message	Date
allenzren	0bdae945e9	use default epoch_start_ema=20 and update_ema_freq=10	2024-11-07 10:55:16 -05:00
allenzren	c0921a1fb5	remove update_ema_freq	2024-11-06 21:01:15 -05:00
Allen Z. Ren	e1ef4ca1cf	More frequent EMA update (#20) ... * move ema update within pretraining epoch * update pretraining ema configs * add lift and can epoch 8000 checkpoint url * add note about EMA issue in pretraining instruction	2024-11-06 20:42:31 -05:00
allenzren	8293b0936b	release	2024-09-03 21:03:27 -04:00