Introducing Gemma 4 12B: A unified, encoder-free multimodal model

Introducing Gemma 4 12B: A unified, encoder-free multimodal model

• 来源：Google DeepMind
• 原文链接：https://blog.google/innovation-and-ai/technology/developers-tools/introducing-gemma-4-12b
• 日期：2026-06-03
• 抓取时间：2026-06-04 20:25

English

Today, we are introducing Gemma 4 12B, our latest model designed to bring agentic multimodal intelligence directly to laptops. Bridging the gap between our edge-friendly E4B and our more advanced 26B Mixture of Experts (MoE), Gemma 4 12B packages powerful capabilities inside a reduced memory footprint. It is also our first mid-sized model to feature native audio inputs.

Thanks to the developer community, Gemma 4 models have now crossed 150 million downloads. You've built everything from wearable robotic arms for physical assistance to enterprise-grade AI security. We're excited to see what you build with this latest addition.

Here's an overview of what makes Gemma 4 12B unique:

• Novel unified architecture: No multimodal encoders. The vision and audio inputs flow directly into the LLM backbone.
• Advanced reasoning: Benchmark performance nearing our 26B model, unlocking powerful multi-step reasoning and agentic workflows.
• Laptop ready: Small enough to run locally with just 16GB of VRAM or unified memory.
• Open and accessible: Released under an Apache 2.0 license with support across the developer ecosystem.
• Drafter-ready: Gemma 4 12B comes equipped with Multi-Token Prediction (MTP) drafters to reduce latency.

Together, these features bring advanced multimodal capabilities to everyday hardware without sacrificing speed or reasoning.

Run state-of-the-art agents locally

Gemma 4 12B delivers performance nearing our larger 26B MoE model on standard benchmarks, but at less than half the total memory footprint. Small enough to run locally on consumer laptops with 16GB of RAM, it unlocks powerful multimodal and agentic experiences right on your machine.

Experience a uniquely efficient, unified architecture

What makes Gemma 4 12B stand out is its streamlined approach to processing visual and audio inputs. Traditional multimodal models typically rely on separate encoders to translate images and audio before passing those representations to the language model. Because these split encoders add latency and increase memory usage, we trained Gemma 4 12B with an encoder-free architecture to integrate audio and vision input directly.

Here is how Gemma 4 12B processes multimodal inputs natively:

• Vision: We replaced Gemma 4's vision encoder with a lightweight embedding module consisting of a single matrix multiplication, positional embedding and normalizations. This allows the LLM backbone to take over visual processing.
• Audio: We simplified audio processing even further. We removed the audio encoder entirely and projected the raw audio signal into the same dimensional space as text tokens.

Get started today

• Try it yourself: Experiment with a couple of clicks in LM Studio, Ollama, Google AI Edge Gallery App, the Google AI Edge Eloquent app and the LiteRT-LM CLI.
• Download the weights: Download the pre-trained and instruction-tuned checkpoints directly from Hugging Face and Kaggle.
• Integrate & learn: Review the developer documentation and the quick start notebook.
• Use your favorite development tools: Implement local inference pipelines with Hugging Face Transformers, llama.cpp, MLX, SGLang, and vLLM, or fine-tune with efficiency using Unsloth.
• Unlock Agentic Development with Gemma Skills: To support agents to build with the latest Gemma advancements, we are releasing our official Skills Repository. This is a library of skills designed specifically to enable agents to build with Gemma models.
• Deploy your way: Spin up endpoints in production using Google Cloud. Deploy your way through Gemini Enterprise Agent Platform Model Garden, Cloud Run and GKE.

中文

今天，我们推出 Gemma 4 12B——一款将"具备智能体能力的多模态智能"直接带到笔记本上的最新模型。它在面向边缘的 E4B 和更先进的 26B 混合专家（MoE）模型之间架起一座桥梁，把强大能力装进更小的内存占用。它也是我们首款原生支持音频输入的中等规模模型。

感谢开发者社区，Gemma 4 全系列模型的下载量现已突破 1.5 亿次。大家基于它构建了从"用于物理辅助的可穿戴机械臂"到"企业级 AI 安全"在内的各种应用。我们非常期待看到你用这款最新模型做出什么。

下面是 Gemma 4 12B 之所以独特的几个关键点：

• 新颖的统一架构：不再有多模态编码器。视觉与音频输入直接汇入 LLM 主干。
• 更强的推理能力：基准表现接近我们的 26B 模型，解锁更强的多步推理与智能体工作流。
• 笔记本即可运行：小到只需 16GB 显存或统一内存即可在本地运行。
• 开放且易用：以 Apache 2.0 协议开源，兼容整个开发生态。
• Drafter-ready：Gemma 4 12B 自带多 token 预测（MTP）drafter，可降低推理延迟。

上述能力加在一起，把先进的多模态能力带到了日常硬件上，又不牺牲速度与推理质量。

在本地运行最先进的 Agent

Gemma 4 12B 在标准基准上的表现接近更大的 26B MoE 模型，但总内存占用不到后者的一半。它小到可以在仅有 16GB 内存的消费级笔记本上本地运行，让强大的多模态和智能体体验直接发生在你的机器上。

体验独特高效、统一化的架构

Gemma 4 12B 之所以与众不同，在于它对视觉和音频输入的处理方式极为简洁。传统多模态模型通常依赖独立的编码器先把图像和音频翻译成表征，再送进语言模型。拆开的编码器会带来延迟、增加内存占用，因此我们用无编码器架构训练了 Gemma 4 12B，让音频和视觉输入直接整合进来。

下面是 Gemma 4 12B 原生处理多模态输入的方式：

• 视觉：我们用一套轻量级的嵌入模块替换了 Gemma 4 的视觉编码器，该模块只包含一次矩阵乘法、位置嵌入和归一化，让 LLM 主干接管视觉处理。
• 音频：我们把音频处理进一步简化——完全去掉音频编码器，把原始音频信号投影到与文本 token 相同的维度空间中。

立即上手

• 亲自试一下：在 LM Studio、Ollama、Google AI Edge Gallery App、Google AI Edge Eloquent App 以及 LiteRT-LM CLI 中点几下即可开始体验。
• 下载权重：从 Hugging Face 和 Kaggle 直接下载预训练与指令微调后的检查点。
• 集成 & 学习：阅读开发者文档与快速上手 notebook。
• 搭配你熟悉的开发工具：用 Hugging Face Transformers、llama.cpp、MLX、SGLang 与 vLLM 搭起本地推理流水线，或用 Unsloth 进行高效微调。
• 用 Gemma Skills 解锁智能体开发：为了支持 Agent 用上最新的 Gemma 能力，我们发布官方的 Skills 仓库——一个专门用来让 Agent 基于 Gemma 模型进行构建的技能库。
• 按需部署：通过 Google Cloud 在生产环境中启动端点；可以通过 Gemini Enterprise Agent Platform Model Garden、Cloud Run 与 GKE 等多种方式部署。