Navigating the Multimodal RAG Revolution: A Breakdown of NVIDIA's Nemotron 3.5 Embed 1B

Scope and intent. This analysis examines NVIDIA's Nemotron 3.5 Embed 1B as a retrieval component for enterprise agents. It covers what the model does, how it was trained, how it performs on published benchmarks, and how it fits into a broader retrieval pipeline. It is an engineering assessment, not a vendor endorsement; benchmark figures are reported as published by NVIDIA and should be re-validated against your own corpora before adoption.

Abstract

Retrieval-augmented generation (RAG) has spent most of its short history as a text-only discipline. Documents were flattened to text, chunked, embedded, and searched. That worked while the answers lived in prose. It breaks down when the answer lives in a chart, a table rendered as an image, an infographic, a datasheet, or a schematic — content whose meaning is carried by layout and pixels, not by an extractable text layer.

NVIDIA's Nemotron 3.5 Embed 1B, released on 20 October 2026, is a direct response to that failure mode. It is an open-weight, 1.55-billion-parameter, encoder-only embedding model that encodes document pages as text, as page images, or as both, and accepts text queries. It was distilled from an 8B teacher and retains 95.4% of that teacher's visual retrieval quality while fitting into a much smaller serving footprint. On the ViDoRe v3.1 dense-retrieval benchmark, the 1B model scores 58.38 nDCG@10 against the 8B model's 61.17 — a result that places the small model within striking distance of the best dense retriever NVIDIA has published.

The more consequential story is not a single benchmark number. It is that a sub-2B multimodal embedding model is now good enough, fast enough, and cheap enough to index enterprise document collections at scale. With an NVFP4 variant delivering 27.1% higher throughput than BF16 on NVIDIA GB200 while retaining 99.85% of retrieval accuracy, and an NVIDIA NIM deployment path hitting 6,031 query requests per second on a single RTX PRO 6000 Blackwell Server Edition GPU, the economics of visual RAG have shifted. This article unpacks the model, its architecture, its benchmark profile, its deployment options, and the retrieval stack it anchors.

1. The multimodal retrieval problem

An agent's answer quality is bounded by the evidence it can retrieve. When that evidence is incomplete or irrelevant, the agent does what any under-informed reasoner does: it searches again. Each retry burns inference tokens, and at enterprise scale that waste compounds into a line item. NVIDIA frames the problem in operational terms — better retrieval lets agents reach grounded answers with fewer wasted tokens — but the root cause is a representation gap.

Consider a concrete task: identify products at risk of stockouts next quarter and explain why. The evidence is scattered across demand forecasts rendered as charts, inventory levels in tables, and supplier lead times in reports. A text-only retriever sees the report text and misses the chart's trend line and the table's numeric relationships. The agent then either answers from partial evidence or keeps searching for text that does not exist.

The industry's first workaround was to run optical character recognition (OCR) and document parsing over every page, then embed the extracted text. That helps when the page has a clean text layer. It fails when:

• The text layer is absent entirely — scanned documents and image-based PDFs.
• Reading order is lost, so a table's values detach from their headers.
• Information is conveyed only visually — a chart's shape, a schematic's topology, an infographic's hierarchy.
• OCR introduces errors that corrupt the very numbers the query depends on.

The alternative is to embed the page image directly, so the retrieval representation preserves layout, typography, and visual structure. That is the approach Nemotron 3.5 Embed takes. It does not replace parsing; it complements it. The model can encode text, image, or a combined representation, which lets a pipeline index whichever modality carries the signal.

2. What Nemotron 3.5 Embed 1B is

Nemotron 3.5 Embed 1B is part of NVIDIA's Nemotron Retriever collection (formerly Nemotron RAG), a family of open, commercially available models for embedding, reranking, and document extraction. The 1B model sits below an 8B sibling and above the task-specific Parse model. The relevant specifications:

The "1B" label is a rounding convention; the model card and reference trackers list 1.55B parameters. The important architectural detail is the split: a Pixtral vision encoder handles page images, while the language component handles text. The language component is initialized from the text-only Nemotron 3 Embed 1B and remains frozen throughout embedding-model training. NVIDIA states that text-only embeddings are therefore expected to remain unchanged from the corresponding Nemotron 3 Embed models, which means existing text embeddings and vector indexes can be reused rather than rebuilt.

That design choice is operationally significant. Teams with an existing text RAG index do not face a full re-index when they add visual retrieval. They can keep the text index and add a parallel image or combined-modality index using the same model family, then fuse results at query time.

2.1 BF16 versus NVFP4

NVIDIA ships two weight formats for the 1B model:

• Nemotron 3.5 Embed 1B BF16 — the reference precision, used for quality-focused evaluation and as the distillation target comparison.
• Nemotron 3.5 Embed 1B NVFP4 — applies FP4 quantization to both the text and image encoders. NVIDIA positions it for large-scale document indexing and high-throughput retrieval on supported hardware.

The NVFP4 variant is the one that changes the production calculus. Quantizing both encoders is harder than quantizing a text-only model because the vision path is more sensitive to precision loss. NVIDIA reports that the quantized model retains 99.85% of the BF16 model's retrieval accuracy on ViDoRe v3 with combined image and text inputs — effectively no quality loss at the benchmark level — while delivering the throughput gain discussed in §6.

3. Architecture and training

Nemotron 3.5 Embed 8B, the teacher model, builds on Ministral-3-8B-Instruct, with its language component initialized from Nemotron-3-Embed-8B-BF16. The model is trained on image and text corpora using contrastive learning to produce representations for multimodal retrieval. The 1B models are distilled from that 8B teacher, with their language component initialized from Nemotron-3-Embed-1B-BF16.

The distillation objective combines cosine and mean squared error (MSE) losses to align the student's embeddings with the teacher's. Cosine loss pushes directional agreement; MSE loss constrains magnitude. Using both is a common pattern when the downstream metric is a similarity score rather than a classification label, because the student needs to match both the teacher's ranking behavior and its embedding geometry.

3.1 Why freezing the language component matters

Because the language component stays frozen during embedding-model training, the text representation is inherited rather than retrained. The vision encoder and the projection that aligns it to the language space are what the multimodal training adapts. This has three consequences:

1. Index reuse. Text embeddings remain compatible with the prior Nemotron 3 Embed generation, so a migration to the 3.5 family does not invalidate an existing text vector store.
2. Reduced training surface. Only the vision path and alignment layers need to learn, which is cheaper and less prone to catastrophic forgetting of text capability.
3. A clean ablation story. Any quality difference between the text-only and combined-modality results is attributable to the visual path, not to a changed language encoder.

The 8B teacher is trained on a mixture of publicly available and synthetic multilingual, multimodal data, and the final released weights are produced through checkpoint ensembling. NVIDIA points to each model's Hugging Face model card for further training-data details. For defenders and procurement teams, the key fact is the license: OpenMDW 1.1 permits commercial use, which removes the legal ambiguity that has slowed adoption of some research-only embedding models.

4. Benchmark performance

Benchmarks are a starting point, not a verdict. The relevant question for an enterprise is whether a model's published results transfer to your documents, languages, and query distribution. With that caveat stated, the published numbers establish where Nemotron 3.5 Embed sits relative to the field.

4.1 ViDoRe v3.1: dense retrieval

ViDoRe v3.1 is the visual document retrieval benchmark that combines image-only, text-only, and combined image-and-text inputs. On it, Nemotron 3.5 Embed 8B BF16 scores 61.17 nDCG@10, ranking first among dense retrieval models as of 20 October 2026. That is an 8.94-point improvement over the 52.23 scored by Llama Nemotron Embed VL 1B.

The 1B model, distilled from the 8B teacher, scores 58.38 nDCG@10 — retaining 95.4% of the teacher's score. The practical reading: the small model captures most of the visual retrieval quality at roughly one-eighth the parameter count. For teams balancing quality against serving cost, that is the central trade-off the model is designed to offer.

4.2 Multilingual visual retrieval

NVIDIA evaluated text retrieval across 34 languages and visual retrieval on MIRACL Vision (18 languages) and KoViDoRe. On MIRACL Vision, the 8B model achieves 72.6 nDCG@10, exceeding Qwen3-VL-Embedding-8B by 4.3 points. On KoViDoRe, both the 8B and 1B models outperform the Qwen baselines when using combined image and text inputs.

The multilingual capability is not incidental. Enterprise document collections are rarely monolingual, and visual documents — invoices, forms, product sheets — often mix languages within a single page. A model that degrades sharply outside English forces teams to maintain per-language indexes. The published results suggest Nemotron 3.5 Embed is intended to avoid that fragmentation, though the usual caveat applies: evaluate on your actual language mix.

4.3 Agentic retrieval and reranking

Embedding quality matters most in the context of an agent loop. NVIDIA evaluated the models on BRIGHT and BrowseComp-Plus (text-only) alongside ViDoRe v3 (combined inputs), using an agent that queries a corpus, inspects evidence, and refines its next search. The retrieval configuration varied while the LLM, tools, task set, corpus, and execution policy stayed fixed.

Two results stand out:

• Lower agent inference cost. Adding Nemotron 3.5 Rerank reduced agentic token cost per query by approximately 14% with the 8B embedder and 13% with the 1B embedder, while improving retrieval accuracy in both configurations.
• Better evidence at the same budget. With the 8B embedder, the 8B reranker delivered approximately 0.9 points higher nDCG@10 than Qwen3-VL-Reranker-8B at a similar agentic token cost.

On the new ViDoRe v3.1 reranking leaderboard, Nemotron 3.5 Rerank 8B BF16 achieves 60.49 nDCG@10 with combined image-and-text inputs, versus Qwen-3-VL-Reranker-8B's 59.17. The reranker is a cross-encoder: it jointly encodes each query-document pair rather than encoding them separately, which is more expensive per pair but more accurate at distinguishing broadly related content from evidence that directly answers the question.

4.4 Benchmark summary

5. Deployment: NVFP4 and NIM

Retrieval quality is only half of the production equation. As document collections and query volumes grow, teams must balance accuracy against throughput and infrastructure cost. Nemotron 3.5 Embed addresses this with two deployment levers: quantization and serving infrastructure.

5.1 NVFP4 throughput

On NVIDIA GB200, the NVFP4 variant delivers 27.1% higher throughput than the 1B BF16 model, increasing from 101.3 to 128.8 documents per second, while retaining 99.85% of retrieval accuracy on ViDoRe v3 with combined image and text inputs. The accuracy-throughput trade-off is effectively flat at the benchmark level.

This matters for indexing economics. Embedding a large document collection is a batch operation whose cost is dominated by throughput; a 27% gain translates directly into fewer GPU-hours. The NVFP4 variant also targets high-throughput query serving, where the same quantization benefit applies to online traffic.

5.2 NIM serving performance

NVIDIA NIM provides a production deployment path for the model. On an RTX PRO 6000 Blackwell Server Edition, NIM delivered 6,031 query requests per second — approximately 2.9× the throughput of a comparable vLLM configuration at 2,099 requests per second — while maintaining comparable passage-processing throughput. For teams already standardized on NIM, that is a straightforward capacity argument; for teams on vLLM, it is a reason to benchmark both before committing.

5.3 Deployment options

The models are available through several channels:

• Open weights on Hugging Face, with training recipes and data details in the model cards.
• NVIDIA NIM for production inference, via the NVIDIA build catalog.
• Inference partners including Baseten, DeepInfra, FriendliAI, and OpenRouter, which host Embed 1B, Embed 8B, and Rerank 8B.
• Elastic customers can access the embedding and reranking models through Elasticsearch's Inference API using NVIDIA inference endpoints; Elastic Inference Service support is planned.

6. The Embed/Rerank/Parse stack

Nemotron 3.5 Embed 1B is not a standalone product; it is one of three roles in a retrieval pipeline. Understanding how the pieces fit clarifies where the 1B model belongs.

6.1 The three roles

6.2 Parse: turning pixels into structure

Nemotron 3.5 Parse addresses the preprocessing gap. Enterprise knowledge often arrives in PDFs whose embedded text captures only part of the page. Extracting that text can lose reading order, separate table values from headers, or omit chart information entirely. Scanned documents may have no usable text layer at all.

Parse converts document images into machine-readable text while preserving structure. It identifies document elements, their locations, and reading order, and extracts tables and charts into structured representations. The output — Markdown, HTML tables, structured blocks — is better suited to indexing and downstream processing. At 631 million parameters, it ranks #1 on ParseBench among models under 1 billion parameters, scoring 75.20 while processing 10.5 pages per second on one H100.

Parse offers three processing modes that trade throughput for accuracy:

All three configurations are Pareto optimal in NVIDIA's comparison: no other configuration achieves both higher accuracy and higher throughput. Teams can pick the operating point that fits their workload, from rapid ingestion of large collections to detailed extraction of complex pages.

6.3 Composing the pipeline

A typical enterprise pipeline combines the three roles: Parse extracts structured content where the text layer is weak or absent; Embed indexes text, page images, or both; Rerank refines the candidate set before it enters the agent's context. The 1B embedder is the cost-efficient indexer and query encoder, while the 8B embedder and 8B reranker handle quality-critical stages. The choice between them is a capacity decision, not an architectural one — both produce embeddings in the same space.

7. Domain adaptation

General-purpose retrieval models encounter enterprise documents full of specialized terminology and company-specific information. Fine-tuning adapts the model to those documents, queries, and relevance labels. NVIDIA provides a multimodal embedding fine-tuning recipe that combines synthetic data generation, hard-negative mining, fine-tuning with NVIDIA NeMo AutoModel, and evaluation in one workflow.

NVIDIA evaluated the recipe on 8,202 chart and infographic images using 8,591 synthetic queries generated with Qwen3.8-Flash-Next, focusing on enumerative and open-ended queries with multiple relevant images. Fine-tuning the 1B model on 6,872 queries for one epoch increased nDCG@10 from 37.34 to 44.12 — an 18% relative gain over the base model on 1,719 held-out synthetic queries. Fine-tuning the 8B model on the same query set for one epoch increased nDCG@10 from 41.44 to 42.75, a 3% relative gain.

The asymmetry is instructive. The 1B model gains far more from domain adaptation than the 8B model does. A plausible explanation is headroom: the larger model already captures more of the domain signal zero-shot, so fine-tuning has less to add. For teams with domain-specific visual documents and a limited budget, the combination of a smaller base model plus fine-tuning may deliver better value than a larger base model alone. NVIDIA's own numbers support that reading, though the experiment used synthetic queries and a single epoch, so production results will vary.

8. Enterprise adoption

NVIDIA reported early evaluations from several enterprise partners. These are vendor-sourced and should be treated as directional:

• Elastic exposes the embedding and reranking models through Elasticsearch's Inference API using NVIDIA inference endpoints. Nemotron 3 will be available via Elastic Inference Service soon, with on-prem support on the roadmap.
• Vectorize evaluated Nemotron 3.5 Embed with Hindsight Agent Memory. It ranked first on two long-conversation memory benchmarks: 93.8% recall@10 on LoCoMo and 65.9% on BEAM 100K against seven other embedding models. It also delivered the highest MRR and recall@5.
• Perplexity reported strong results on Q2D-Web, its benchmark for first-stage retrieval in agentic search. On its private 72-task reranking benchmark, Nemotron 3.5 Rerank 8B achieved the highest average score among the rerankers evaluated, with strong results across all six domains.
• Turbopuffer evaluated Nemotron 3.5 Embed 8B on its enterprise PDF search benchmark and found approximately 5% more relevant results at Recall@20 than the next best model. The models are available in Turbopuffer's native embedding.

The Vectorize result is the most interesting for agent builders. Long-conversation memory is a retrieval problem where the evidence is spread across many turns and the query is often implicit. A model that performs well there suggests the embedding generalizes beyond document pages to conversational context — a use case NVIDIA did not emphasize but one that matters for persistent agents.

9. Implications for multimodal RAG

Three takeaways stand out for teams building retrieval systems.

9.1 Visual retrieval is now a commodity capability

A 1.55B open-weight model scoring 58.38 nDCG@10 on ViDoRe v3.1, with a quantized variant that retains 99.85% of accuracy at 27% higher throughput, changes the build-versus-buy calculus. The barrier to adding visual retrieval to an existing RAG pipeline is now mostly engineering integration, not model access or licensing. OpenMDW 1.1 removes the commercial-use ambiguity that has slowed adoption of research-only models.

9.2 The retrieval stack matters more than the embedder

NVIDIA's own agentic evaluation shows that adding a reranker reduces token cost by 13-14% while improving accuracy. The embedder determines what is retrievable; the reranker determines what reaches the context window. Teams optimizing only the embedding model while ignoring reranking leave measurable efficiency on the table. The Parse model adds a third lever: better preprocessing means better embeddings, especially for documents whose text layer is weak.

9.3 Evaluate on your own corpus

Benchmark leadership on ViDoRe v3.1 does not guarantee leadership on your documents. The published results use specific corpora, languages, and query distributions. Enterprise collections have their own vocabulary, layout conventions, and relevance criteria. NVIDIA's fine-tuning recipe exists precisely because the base model is a starting point. The 18% relative gain on chart and infographic data is a useful signal that domain adaptation is worth the effort when your documents are specialized.

9.4 A note on retrieval security

Retrieval models are part of the attack surface. A multimodal embedder that ingests page images can be exposed to adversarial documents designed to manipulate retrieval rankings or inject instructions into downstream context. The same properties that make visual retrieval powerful — accepting pixels as input — make it harder to sanitize than text. Teams deploying visual RAG should treat retrieved documents as untrusted input and apply the same content-inspection controls they would to any external data source. This article does not assess the model's robustness to adversarial documents; that is a separate evaluation.

10. References

1. NVIDIA, Nemotron 3.5 Embed, Rerank, and Parse: Advancing Agents with World-Class Retrieval Models. Hugging Face blog, 21 October 2026. https://huggingface.co/blog/nvidia/nemotron-retriever-world-class-agentic-retrieval
2. NVIDIA, Nemotron-3.5-Embed-1B-BF16 model card. https://huggingface.co/nvidia/Nemotron-3.5-Embed-1B-BF16
3. NVIDIA, Nemotron-3.5-Embed-1B-NVFP4 model card. https://huggingface.co/nvidia/Nemotron-3.5-Embed-1B-NVFP4
4. NVIDIA, Nemotron-3.5-Embed-8B-BF16 model card. https://huggingface.co/nvidia/Nemotron-3.5-Embed-8B-BF16
5. NVIDIA, Nemotron-3.5-Rerank-8B-BF16 model card. https://huggingface.co/nvidia/Nemotron-3.5-Rerank-8B-BF16
6. NVIDIA, Nemotron-3.5-Parse model card. https://huggingface.co/nvidia/Nemotron-3.5-Parse
7. NVIDIA, Nemotron Retriever collection. https://huggingface.co/collections/nvidia/nemotron-retriever
8. MTEB Leaderboard, ViDoRe v3.1 benchmark. https://mteb-leaderboard.hf.space/benchmark/ViDoRe(v3.1)
9. MTEB Leaderboard, ViDoRe v3.1 Reranking benchmark. https://leaderboard.mteb.org/benchmark/ViDoRe(v3.1,%20reranking)
10. NVIDIA, Nemotron multimodal embedding fine-tuning recipe. https://github.com/NVIDIA-NeMo/Nemotron/tree/main/src/nemotron/recipes/embed
11. NVIDIA, Nemotron reranking fine-tuning recipe. https://github.com/NVIDIA-NeMo/Nemotron/tree/main/src/nemotron/recipes/rerank
12. LLM Reference, Nemotron 3.5 Embed 1B — 32K context, multimodal, open weights. https://www.llmreference.com/model/nemotron-3.5-embed-1b
13. OpenMDW 1.1 license. https://openmdw.org/licenses/openmdw/1.1/
14. MTEB, BRIGHT benchmark. https://brightbenchmark.github.io/
15. Hugging Face Spaces, BrowseComp-Plus. https://huggingface.co/spaces/Tevatron/BrowseComp-Plus

Prepared as a technical assessment of a publicly released model. Benchmark figures are as published by NVIDIA and partner organizations; independent validation on your own corpora is recommended before adoption. The model is released under OpenMDW 1.1, which permits commercial use. Verify current model card details, license terms, and benchmark status before deploying.