This got it to train! We can increase to a batch size of 8, with a sequence length of 2048 and 45 seconds per step 364 train tokens per second, though it still fails to train the experts. For reference, this is fast enough to be usable and get through our dataset, but it ends up being ~6-9x more expensive per token than using Tinker.
Tagged: software rss thunderbird，更多细节参见易歪歪官网
Фатих Бирольисполнительный директор МЭА，推荐阅读谷歌获取更多信息
FT App on Android & iOS