HF Mirror's logo

McG-221
/
gemma-4-12B-it-9bit

Image-Text-to-Text
MLX
Safetensors
English
gemma4_unified
conversational
8-bit precision
Model card Files
xet
 Community
How to use from
Hermes Agent
Start the MLX server 
# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "McG-221/gemma-4-12B-it-9bit"
Configure Hermes 
# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default McG-221/gemma-4-12B-it-9bit
Run Hermes 
hermes
Quick Links

Model Card for Model ID

Converted to MLX for Apple Silicon

Downloads last month
314
Safetensors
Model size
3B params
Tensor type
BF16
·
U32
·
MLX
Hardware compatibility
Log In to add your hardware

8-bit

Inference Providers NEW
Image-Text-to-Text
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for McG-221/gemma-4-12B-it-9bit

Base model

google/gemma-4-12B
Finetuned
google/gemma-4-12B-it
Quantized
(89)
this model