"OLMo 2 [was] developed start-to-finish with open and accessible training data, open-source training code, reproducible training recipes, transparent evaluations, intermediate checkpoints, and more," AI2 wrote in a blog post.
"By openly sharing our data, recipes, and findings, we hope to provide the open-source community with the resources needed to discover new and innovative approaches."
"To train the models, Ai2 used a dataset of 5 trillion tokens. The training set included websites 'filtered for high quality,' academic papers, and Q&A discussion boards."
Collection
[
|
...
]