The Allen Institute for AI (AI2), a non-profit research organization founded in 2014 by the late Microsoft co-founder Paul Allen, has introduced OLMo, described as a cutting-edge large language model that is truly open-source. This marks a notable progression in AI advancement by presenting an alternative to existing closed models.
In contrast to previous models that solely provided the model code and weights, OLMo sets a new standard by offering not only the model code and weights but also the training code, training data, toolkits, and evaluation resources. Additionally, it is released under the Apache 2.0 License, which is endorsed by the open-source initiative (OSI).
This unveiling comes at a crucial time when open-source AI, which has historically trailed behind closed proprietary models such as OpenAI’s GPT-4 and Anthropic’s Claude, is gaining momentum. Recent developments include a Paris-based open-source AI startup introducing a model that rivals GPT-4’s performance. Meta has also enhanced its code generation model with Code Llama 70B, while the anticipation for the next version of its Llama LLM continues to grow.
Despite these advancements, open-source AI encounters criticism from various stakeholders, including researchers, regulators, and policymakers. The ongoing discourse surrounding the risks and benefits of AI applications will be a key focus at the upcoming AI Impact Tour in New York on February 29, organized jointly with Microsoft.
The OLMo framework offers an extensive set of AI development tools to the public, covering pretraining data, training code, model weights, and evaluation resources. It encompasses inference code, training metrics, logs, and an evaluation suite with over 500 checkpoints per model. AI2 researchers are committed to further refining OLMo by exploring various model sizes, datasets, and functionalities.
Hanna Hajishirzi, the lead of the OLMo project and senior director of NLP Research at AI2, stressed the significance of transparency in language models, likening the absence of access to training data to conducting drug discovery without clinical trials. Nathan Lambert, an ML scientist at AI2, underscored OLMo’s exceptional openness, enabling researchers to delve deeper into the science of LLMs.
The release of OLMo has garnered praise from the open-source AI community, with Jonathan Frankle from MosaicML and Databricks hailing it as “a significant stride for open science.” Yann LeCun, Meta’s chief scientist, lauded the role of open foundation models in promoting innovation and progress in generative AI.
In summary, the debut of OLMo signifies a noteworthy achievement in open-source AI, granting researchers unparalleled access and resources to propel ML research and implementation forward.
