Silo AI, a startup specializing in artificial intelligence located in Helsinki, Finland, gained attention this week by unveiling Poro, a new open-source large language model (LLM) aimed at enhancing multilingual AI capabilities for European languages.
Poro, the initial installment in a series of planned open-source models, is set to eventually encompass all 24 official European Union dialects. Developed by SiloGen in collaboration with the TuruNLP research team at the University of Turku and the conceptual AI division founded in late 2022.
Peter Sarlin, the CEO of Silo AI, emphasized the importance of incorporating cultural values, language nuances, and digital sovereignty in the design process. Speaking with VentureBeat, Sarlin highlighted the significance of creating value not only for Europeans but for any organization seeking to leverage custom models within the European framework.
Named after the Finnish term for “reindeer,” the Poro 34B model boasts a 34.2 billion feature capacity utilizing a BLOOM converter architecture with ALiBi embeddings. Its training data, sourced from a vast 21 trillion multicultural database covering English, Scandinavian languages, as well as programming languages like Python and Java, was harnessed for its development.
Poro is undergoing training on LUMI, Europe’s fastest supercomputer located in Kajaani, Finland. Powered by 512 AMD Instinct MI250X GPUs delivering 74 petaflops of processing capability, LUMI enables advanced training for Poro.
The primary objective of Poro, according to Sarlin, is to address the challenge of creating effective natural language models for less-resourced languages such as Finnish. This is achieved through a cross-lingual training approach that leverages insights from more linguistically rich languages like English.
In the wake of Mistral AI’s unveiling of Misral 7B in late September 2023, Poro emerges as a significant open-source LLM originating from Europe. This development underscores the continent’s advancements in the rapidly evolving field of generative AI and underscores the escalating competition among various AI research labs and companies.
Poro Research Milestones
As part of SiloGen’s commitment to transparency, the Checkpoints for Poro Research program will document Poro’s training progress.
Sarlin highlighted the novelty of providing checkpoints throughout the model’s training. He stated, “Model training hasn’t seen this level of transparency before.”
The initial checkpoint for Poro 34B covers the first 30% of the training process. Reports from Silo AI indicate that even at this early stage, Poro continues to deliver cutting-edge results.
Poro has demonstrated superior performance compared to existing bilingual Finnish AI models like FinGPT on the widely recognized FIN-bench evaluation for the Finnish language.
Sarlin noted, “Even after 30% of the training, the model exhibits enhanced capabilities for low-resource languages.” By leveraging shared linguistic patterns across related languages, Poro offers a competitive edge for languages with limited training data.
Remarkably, Poro’s bilingual proficiency does not compromise its command of the English language. Sarlin affirmed that the model already surpasses existing benchmarks for Scandinavian languages and matches the performance standards for English.
An Alternative to Big Tech through Open Source
Sarlin envisions open-source models like Poro as a transparent and ethical alternative to proprietary models from tech giants, heralding a new era for AI.
He expressed his belief that “a plethora of open-source alternatives will eventually emerge.” By embracing open-source principles, organizations can gain full visibility into the model’s development process and architecture.
Silo AI plans to continue releasing regular Poro checkpoints during the training phase. The ultimate goal is to establish a comprehensive suite of open-source models supporting all European languages, potentially challenging the dominance of Big Tech in the AI landscape.
Collaboration with the University of Turku
The collaboration between Silo AI and the University of Turku remains pivotal to Poro’s development. The TurkuNLP research group at the university played a key role in creating open-source tools and models for the Estonian language.
Sarlin highlighted the unique composition of their team, comprising over 300 members, predominantly PhD holders in AI-related fields. This partnership leverages the University’s expertise in multicultural language modeling research alongside Silo AI’s computational resources and applied IoT knowledge.
This collaboration exemplifies how academia and industry can synergize to enhance AI capabilities, particularly for underrepresented European languages.
Europe’s Potential as a Leader in Open Source AI
The launch of Poro signifies a shift towards open collaboration and transparency in natural language processing. Initiatives like the Checkpoints for Poro Research program democratize access to tools and insights that were previously confined within tech behemoths.
Sarlin highlighted their collaborations with prominent companies like Allianz, Rolls Royce, Honda, and Philips, emphasizing the growing concerns among enterprises regarding regulatory compliance and model selection.
If Poro fulfills its potential, it could democratize access to effective multilingual models, offering Europe a viable alternative to the AI systems of US tech giants. Despite being in its nascent stages, Poro marks a significant milestone in democratizing language AI and breaking free from proprietary silos.
