On Wednesday, Google introduced a new series of AI language models called Gemma, which are open-weights models similar to the more powerful yet closed Gemini models. In contrast to Gemini, Gemma models can function locally on a desktop or laptop computer. This release signifies Google’s first major launch of an open large language model (LLM) since the rise of OpenAI’s ChatGPT, which triggered an AI chatbot craze in 2022.
The Gemma models come in two variants: Gemma 2B (2 billion parameters) and Gemma 7B (7 billion parameters), both available in pre-trained and fine-tuned versions. Parameters in AI refer to the values in a neural network that influence the model’s behavior, while weights are a subset of these parameters stored in a file.
Developed by Google DeepMind and other Google AI teams, Gemma is inspired by the techniques used in creating Gemini, which encompasses Google’s most advanced publicly available commercial LLMs, including those powering its Gemini AI assistant. The name “Gemma” is derived from the Latin word gemma, meaning “precious stone.”
While Gemma represents Google’s primary open LLM release post-ChatGPT (with previous introductions of smaller research models like FLAN-T5), Google has a track record of contributing to open AI research. The company highlights the evolution of the Transformer architecture and the releases of TensorFlow, BERT, T5, and JAX as significant contributions that have shaped the field.
Although not strictly “open source,” Google Gemma is seen as a strategic move to compete with Meta, which has focused on releasing open-weights models (e.g., LLaMA and Llama 2) since February last year. This differs from models like OpenAI’s GPT-4 Turbo, which is only accessible through the ChatGPT application and a cloud API without local deployment options. A report by Reuters on Gemma underscores the competitive angle with Meta and suggests that Google aims to attract more developers to its Vertex AI cloud platform.
Despite not yet utilizing Gemma, Google claims that the 7B model outperforms Meta’s Llama 2 7B and 13B models across various benchmarks, including math, Python code generation, general knowledge, and commonsense reasoning tasks. It is currently available on Kaggle, a platform in the machine-learning community, and Hugging Face.
In a related development, Google also introduced the Gemma release alongside a “Responsible Generative AI Toolkit,” aimed at providing guidance and tools for creating what the company describes as “safe and responsible” AI applications.
