Stockmark Nemotron 3 Nano Omni JapanDocReader

--- license: other license_name: nvidia-open-model-agreement license_link: >- https://www.nvidia.com/en-us/agreements/enterprise-software/nvidia-open-model-agreement/ language: - ja base_model: - nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-BF16 pipeline_tag: image-text-to-text library_name: transformers tags: - multimodal --- # Stockmark-Nemotron-3-Nano-Omni-JapanDocReader ## Introduction **Stockmark-Nemotron-3-Nano-Omni-JapanDocReader** is a Japanese document-reading multimodal model built on top of NVIDIA's [Nemotron-3-Nano-Omni-30B-A3B-Reasoning](https://huggingface.co/nvidia/Nemotron-3-Nano-Omni-30B-A3B-Reasoning-BF16). This model injects Japanese structured document parsing capability while preserving as much of the original VQA ability as possible. It is trained with a two-stage recipe: 1. **Mixed SFT** — supervised fine-tuning on a mixture of structured parsing data and VQA data. Mixing VQA into the SFT set mitigates catastrophic forgetting of the base model's reasoning/VQA ability that pure structured-parsing-only SFT would cause. SFT was performed with [NeMo Megatron-Bridge](https://github.com/NVIDIA-NeMo/Megatron-Bridge). 2. **DAPO** — reinforcement learning applied to the structured document parsing capability on top of the mixed-SFT base. This pushes structured document parsing quality above the SFT base while keeping the VQA regression small. DAPO was performed with [NeMo-RL](https://github.com/NVIDIA-NeMo/RL). All training keeps **thinking mode** on and uses **English reasoning traces**, which is central to preserving multi-step-reasoning VQA performance (switching to no-think or changing the reasoning language degrades the base model's reasoning distribution). Training data was synthesized with Stockmark's [Japanese document synthesis pipeline](https://stockmark-tech.hatenablog.com/entry/2026/04/28/100048) using [Nemotron-Personas-Japan](https://huggingface.co/datasets/nvidia/Nemotron-Personas-Japan) and [Qwen3.6-27B](https://huggingface.co/Qwen/Qwen3.6-27B) as the generator. > For a detailed write-up (in Japanese), see our [our blog](https://stockmark-tech.hatenablog.com/entry/2026/07/16/000000). ## Quickstart The model serves as a standard OpenAI-compatible endpoint via vLLM (≥ 0.20). It is a reasoning model — keep thinking mode on and inject a reasoning budget so it closes `</think>` and answers. ### 1. Launch the server `...

Source context: 977 downloads · 8 likes · Pipeline image-text-to-text · Library transformers · Repo stockmark/Stockmark-Nemotron-3-Nano-Omni-JapanDocReader