*This model was contributed to Hugging Face Transformers on 2026-09-23.*
# NemotronH Omni
NemotronH Omni is a multimodal reasoning model from NVIDIA that pairs the [NemotronH](./nemotron_h) hybrid
Mamba-Transformer language model with a [RADIO](./radio) vision encoder and an optional Parakeet-based sound encoder.
Image (and video) patches are projected through a RADIO tower and a pixel-shuffle MLP into the language model's
embedding space at the `` / `` context-token positions; audio clips are projected in the same way at
`` positions. The result is a single autoregressive model that reasons jointly over text, images, video and
sound.
The example below demonstrates how to reason over an image and a text prompt with [`AutoModelForImageTextToText`].
```python
from transformers import AutoModelForImageTextToText, AutoProcessor
model_id = "nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-BF16"
processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForImageTextToText.from_pretrained(
model_id,
device_map="auto",
# the Parakeet audio encoder has no flash-attention kernel, so pin that tower to sdpa
attn_implementation={"": "flash_attention_2", "audio_config": "sdpa"},
)
messages = [
{
"role": "user",
"content": [
{
"type": "image",
"url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/pipeline-cat-chonk.jpeg",
},
{"type": "text", "text": "Describe this image in detail."},
],
}
]
inputs = processor.apply_chat_template(
messages,
add_generation_prompt=True,
tokenize=True,
return_dict=True,
return_tensors="pt",
).to(model.device)
output = model.generate(**inputs, max_new_tokens=128, do_sample=False)
print(processor.decode(output[0, inputs["input_ids"].shape[-1] :], skip_special_tokens=True))
```
## NemotronH_Omni_Reasoning_V3_Config
[[autodoc]] NemotronH_Omni_Reasoning_V3_Config
## NemotronH_Omni_Reasoning_V3ImageProcessor
[[autodoc]] NemotronH_Omni_Reasoning_V3ImageProcessor
## NemotronH_Omni_Reasoning_V3VideoProcessor
[[autodoc]] NemotronH_Omni_Reasoning_V3VideoProcessor
## NemotronH_Omni_Reasoning_V3Processor
[[autodoc]] NemotronH_Omni_Reasoning_V3Processor
## NemotronH_Omni_Reasoning_V3
[[autodoc]] NemotronH_Omni_Reasoning_V3
- forward
- generate
