Mistral AI, a pioneering open-source startup, unveiled a new LLM today by simply sharing a download link, reminiscent of a surprise device drop.
This unconventional approach immediately sparked interest within the community, especially when compared to Google’s recent Gemini release. Andrej Karpathy from OpenAI criticized Google’s polished launch video as “over-rehearsed” in a cross-post.
Within 24 hours, scrutiny arose over a video segment from Google’s launch, which was deemed as staged and overly edited to project a sense of proficiency.
In contrast, Mistral opted for a minimalist approach by releasing a substantial download file for their latest model, MoE8x7B.
The Mistral LLM was dubbed a “scaled-down GPT-4” and described as a “MoE comprising 8 7B authorities” in a Reddit post. The model reportedly employs only two experts for each token’s inference. Speculations suggest that GPT-4 may be a MoE model with 8 experts, each possessing 111B parameters individually and 55B shared attention parameters (166). Similarly, only two experts are utilized for each token’s assumption.
Uri Eliabayev, an AI consultant and the mind behind “System & Deep Learning Israel,” highlighted Mistral’s reputation for such low-key releases, devoid of formal reports, websites, code, or media announcements. Jay Scambler, an advocate for open-source AI, described the move as “unconventional” yet undeniably successful in generating excitement.
The bold move by Mistral garnered praise from various quarters within the AI community. Renowned innovator George Hotz commended the initiative, while Eric Jang, Vice Chairman of AI at 1X Technologies and a former robotics research scientist at Google, expressed his growing admiration for Mistral’s product.
Mistral, headquartered in Paris, recently secured a remarkable \(2 billion in funding led by Andreessen Horowitz, following its groundbreaking \)18 million seed round, touted as the largest in European history. The company made waves with the launch of its first large-language AI model, the Mitral 7B, back in September.
Reports indicate that Mistral has been actively lobbying the European Parliament for more lenient regulations on open-source AI, placing the company at the forefront of discussions surrounding the EU AI Act.
VentureBeat aims to serve as a contemporary hub where industry leaders can engage and explore cutting-edge enterprise technologies.
