RT @YiTayML: If anything, this graph tells me that RedPajama is worse than the Pile (Pythia) at the same number of pretraining tokens.

People can read graphs right? 🥲

But yea few-shot at 3b is probably really just noise tbh anyway. https://t.co/Uam6x64W10