ModelVerifiedExternal

VisionEncoder

Hosted artifacts (derived data + trained checkpoints) for the VisionEncoder research project.

Sourcehuggingface.co/xiaomoguhzz/VisionEncoder/tree/792e9bc2fee763b54517c787a7d400270588c65e

KindCheckpointBase modelQwen3VersionRevision 792e9bcPublisher@xiaomoguhzzCgrade

Model source

Kind: Checkpoint
File size: 9.39 MB–16.5 GB
Base model: Qwen3
Version: vRevision 792e9bc
Updated: Oct 3, 2026
Publisher: @xiaomoguhzz
Downloads: 0
Parameters: 4B
Source: Hugging Face

Source description

VisionEncoder

Hosted artifacts (derived data + trained checkpoints) for the VisionEncoder research project.

Training code + full reproduction guide: https://github.com/xiaomoguhz/VisionEncoder

The repo is organized into three top-level folders.

`data/` — current (V9.x) reproduction data (~6.5G)

Path	Content
`data/vmllm_cached/qwen3vit/`	S2 `cached_dataset` arrow (image/video, 10pct + full); fed directly to stage-2
`data/ms-swift-data/`	sampled sharegpt jsonl (10pct + full)
`data/llava_video/`	V9 decode-probed `good_manifest` for the video path

`ckpts/` — ready-made 4B MLLM inference weights

Path	Content
`ckpts/4b_stock`	4B stock baseline (raw Qwen3.5 ViT, skips declip), checkpoint-505, 9.5G
`ckpts/4b_v9_1`	4B V9.1 (V-JEPA 2.1 video self-distill), checkpoint-505, 9.5G

Download either and feed it straight to evaluation (see the GitHub README, section 4 — MLLM evaluation) to skip declip + S1 + S2.

`legacy/` — historical assets (~368G)

Early-line products, not needed to reproduce the current main line: declip_siglip2/spatial_align, kd_mllm, self_refine, video_mllm_swift (old SigLIP2 / image-only S1+S2 ckpts), and old ViT-family arrow caches.

Download

# current dev data
huggingface-cli download xiaomoguhzz/VisionEncoder --include "data/*" --local-dir .
# ready-made 4B MLLM ckpt (eval directly)
huggingface-cli download xiaomoguhzz/VisionEncoder --include "ckpts/4b_v9_1/*" --local-dir .

Code + reproduction guide: https://github.com/xiaomoguhz/VisionEncoder

Sources

1 source

Hugging FacePrimaryVersion 792e9bc2fee763b54517c787a7d400270588c65e

huggingface.co/xiaomoguhzz/VisionEncoder/tree/792e9bc2fee763b54517c787a7d400270588c65e

Verified Oct 3

Download files

Model artifacts

64 artifacts

ckpts/S0/qwen3_5_2b/v10_2/step000348/mp_rank_00_model_states.pt

pt · 596 MB · SHA-256 a7f86a4a9fb1…35b5 · Hugging Face

Download file

ckpts/S2/4b/qwen3_5_2b/baseline_16f_10pct/v0-20260530-001435/checkpoint-505/model-00001-of-00002.safetensors

safetensors · 4.64 GB · SHA-256 87bf8644b6e1…2a3c · Hugging Face

Download shard

ckpts/S2/4b/qwen3_5_2b/baseline_16f_10pct/v0-20260530-001435/checkpoint-505/model-00002-of-00002.safetensors

safetensors · 4.16 GB · SHA-256 feb82dce0fbd…7757 · Hugging Face

Download shard

ckpts/S2/4b/qwen3_5_2b/baseline_32f_10pct/v0-20260606-044138/checkpoint-785/model-00001-of-00002.safetensors

safetensors · 4.64 GB · SHA-256 8bb89a0cb17b…af25 · Hugging Face

Download shard

ckpts/S2/4b/qwen3_5_2b/baseline_32f_10pct/v0-20260606-044138/checkpoint-785/model-00002-of-00002.safetensors

safetensors · 4.16 GB · SHA-256 549730da56da…b529 · Hugging Face

Download shard

ckpts/S2/4b/qwen3_5_2b/v10_2_32f_10pct/v0-20260610-154054/checkpoint-785/model-00001-of-00002.safetensors

safetensors · 4.64 GB · SHA-256 128b40150f51…9114 · Hugging Face

Download shard

ckpts/S2/4b/qwen3_5_2b/v10_2_32f_10pct/v0-20260610-154054/checkpoint-785/model-00002-of-00002.safetensors

safetensors · 4.16 GB · SHA-256 4ad7114700d7…fc76 · Hugging Face

Download shard

ckpts/S2/4b/qwen3_5_2b/v13_0/v0-20260817-033352/checkpoint-1530/model-00001-of-00002.safetensors

safetensors · 4.64 GB · SHA-256 833bcc65de33…cb50 · Hugging Face

Download shard

ckpts/S2/4b/qwen3_5_2b/v13_0/v0-20260817-033352/checkpoint-1530/model-00002-of-00002.safetensors

safetensors · 4.18 GB · SHA-256 c6fc19c93f99…6315 · Hugging Face

Download shard

ckpts/S2/4b/qwen3_5_2b/v13_1/v0-20260817-033401/checkpoint-1530/model-00001-of-00002.safetensors

safetensors · 4.64 GB · SHA-256 c94fbd9a6d31…7b3a · Hugging Face

Download shard

ckpts/S2/4b/qwen3_5_2b/v13_1/v0-20260817-033401/checkpoint-1530/model-00002-of-00002.safetensors

safetensors · 4.18 GB · SHA-256 5682f1626f94…a9e9 · Hugging Face

Download shard

ckpts/S2/4b/qwen3_5_2b/v13_2/v1-20260817-033406/checkpoint-1530/model-00001-of-00002.safetensors

safetensors · 4.64 GB · SHA-256 343e726ff3ab…e632 · Hugging Face

Download shard

ckpts/S2/4b/qwen3_5_2b/v13_2/v1-20260817-033406/checkpoint-1530/model-00002-of-00002.safetensors

safetensors · 4.18 GB · SHA-256 563779a225a5…f1cd · Hugging Face

Download shard

ckpts/S2/4b/qwen3_5_2b/v13_3/v0-20260817-190428/checkpoint-1530/model-00001-of-00002.safetensors

safetensors · 4.64 GB · SHA-256 b79b9300f732…c6dc · Hugging Face

Download shard

ckpts/S2/4b/qwen3_5_2b/v13_3/v0-20260817-190428/checkpoint-1530/model-00002-of-00002.safetensors

safetensors · 4.18 GB · SHA-256 f26e1557ca2f…f3e2 · Hugging Face

Download shard

ckpts/S2/4b/qwen3_5_2b/v13_5/v0-20260821-172306/checkpoint-1530/model-00001-of-00002.safetensors

safetensors · 4.64 GB · SHA-256 2b5608c6e998…c103 · Hugging Face

Download shard

ckpts/S2/4b/qwen3_5_2b/v13_5/v0-20260821-172306/checkpoint-1530/model-00002-of-00002.safetensors

safetensors · 4.18 GB · SHA-256 df3e5dc26839…7ebf · Hugging Face

Download shard

ckpts/S2/4b/qwen3_5_2b/v13_6/v0-20260820-192228/checkpoint-1400/model-00001-of-00002.safetensors

safetensors · 4.64 GB · SHA-256 c2a8105e2147…fb0e · Hugging Face

Download shard

ckpts/S2/4b/qwen3_5_2b/v13_6/v0-20260820-192228/checkpoint-1400/model-00002-of-00002.safetensors

safetensors · 4.18 GB · SHA-256 6948df15a852…9e25 · Hugging Face

Download shard

ckpts/S2/4b/qwen3_5_2b/v9_1_16f_10pct/v0-20260602-181529/checkpoint-505/model-00001-of-00002.safetensors

safetensors · 4.64 GB · SHA-256 daecd183bea1…a448 · Hugging Face

Download shard

ckpts/S2/4b/qwen3_5_2b/v9_1_16f_10pct/v0-20260602-181529/checkpoint-505/model-00002-of-00002.safetensors

safetensors · 4.16 GB · SHA-256 c36c9a449fbc…53bd · Hugging Face

Download shard

legacy/declip_siglip2/spatial_align/step011088/bf16_zero_pp_rank_0_mp_rank_00_optim_states.pt

pt · 610 MB · SHA-256 a580164c97ec…6f7c · Hugging Face

Download file

legacy/declip_siglip2/spatial_align/step011088/bf16_zero_pp_rank_1_mp_rank_00_optim_states.pt

pt · 610 MB · SHA-256 53730180460e…2e25 · Hugging Face

Download file

legacy/declip_siglip2/spatial_align/step011088/bf16_zero_pp_rank_2_mp_rank_00_optim_states.pt

pt · 610 MB · SHA-256 23c00a2d7bf3…4728 · Hugging Face

Download file

legacy/declip_siglip2/spatial_align/step011088/bf16_zero_pp_rank_3_mp_rank_00_optim_states.pt

pt · 610 MB · SHA-256 28e7836eb617…a7b6 · Hugging Face

Download file

legacy/declip_siglip2/spatial_align/step011088/bf16_zero_pp_rank_4_mp_rank_00_optim_states.pt

pt · 610 MB · SHA-256 f8092e20ebdd…90bb · Hugging Face

Download file

legacy/declip_siglip2/spatial_align/step011088/bf16_zero_pp_rank_5_mp_rank_00_optim_states.pt

pt · 610 MB · SHA-256 f41cbce6a846…fdfb · Hugging Face

Download file

legacy/declip_siglip2/spatial_align/step011088/bf16_zero_pp_rank_6_mp_rank_00_optim_states.pt

pt · 610 MB · SHA-256 78af89b2b395…1a07 · Hugging Face

Download file

legacy/declip_siglip2/spatial_align/step011088/bf16_zero_pp_rank_7_mp_rank_00_optim_states.pt

pt · 610 MB · SHA-256 25fda71b7853…7323 · Hugging Face

Download file

legacy/declip_siglip2/spatial_align/step011088/mp_rank_00_model_states.pt

pt · 820 MB · SHA-256 8dfef1ba0c9f…6d43 · Hugging Face

Download file

legacy/kd_mllm/s1_kd_pretrain/checkpoint-2181/global_step2181/bf16_zero_pp_rank_0_mp_rank_00_optim_states.pt

pt · 9.39 MB · SHA-256 02b217ad82b0…d9ed · Hugging Face

Download file

legacy/kd_mllm/s1_kd_pretrain/checkpoint-2181/global_step2181/bf16_zero_pp_rank_1_mp_rank_00_optim_states.pt

pt · 9.39 MB · SHA-256 5f24d7ba0296…f3df · Hugging Face

Download file

legacy/kd_mllm/s1_kd_pretrain/checkpoint-2181/global_step2181/bf16_zero_pp_rank_2_mp_rank_00_optim_states.pt

pt · 9.39 MB · SHA-256 ff08572611a1…cf3a · Hugging Face

Download file

legacy/kd_mllm/s1_kd_pretrain/checkpoint-2181/global_step2181/bf16_zero_pp_rank_3_mp_rank_00_optim_states.pt

pt · 9.39 MB · SHA-256 1b40fcb61e95…3026 · Hugging Face

Download file

legacy/kd_mllm/s1_kd_pretrain/checkpoint-2181/global_step2181/bf16_zero_pp_rank_4_mp_rank_00_optim_states.pt

pt · 9.39 MB · SHA-256 533d13e685dd…517a · Hugging Face

Download file

legacy/kd_mllm/s1_kd_pretrain/checkpoint-2181/global_step2181/bf16_zero_pp_rank_5_mp_rank_00_optim_states.pt

pt · 9.39 MB · SHA-256 e34d61ed0af5…9e35 · Hugging Face

Download file

legacy/kd_mllm/s1_kd_pretrain/checkpoint-2181/global_step2181/bf16_zero_pp_rank_6_mp_rank_00_optim_states.pt

pt · 9.39 MB · SHA-256 183c35dd56d9…9ac4 · Hugging Face

Download file

legacy/kd_mllm/s1_kd_pretrain/checkpoint-2181/global_step2181/bf16_zero_pp_rank_7_mp_rank_00_optim_states.pt

pt · 9.39 MB · SHA-256 7d17a6e68667…af28 · Hugging Face

Download file

legacy/kd_mllm/s1_kd_pretrain/checkpoint-2181/global_step2181/mp_rank_00_model_states.pt

pt · 7.90 GB · SHA-256 71860b3d080d…e648 · Hugging Face

Download file

legacy/kd_mllm/s1_kd_pretrain/checkpoint-2181/model.safetensors

safetensors · 4.54 GB · SHA-256 c155c806a542…15d0 · Hugging Face

Download file

legacy/kd_mllm/s1_siglip2_qwen3_4b/v1-20260320-102316/checkpoint-2181/global_step2181/bf16_zero_pp_rank_0_mp_rank_00_optim_states.pt

pt · 13.6 MB · SHA-256 2033c30d00d2…e06c · Hugging Face

Download file

legacy/kd_mllm/s1_siglip2_qwen3_4b/v1-20260320-102316/checkpoint-2181/global_step2181/bf16_zero_pp_rank_1_mp_rank_00_optim_states.pt

pt · 13.6 MB · SHA-256 777247a75c8e…c568 · Hugging Face

Download file

legacy/kd_mllm/s1_siglip2_qwen3_4b/v1-20260320-102316/checkpoint-2181/global_step2181/bf16_zero_pp_rank_2_mp_rank_00_optim_states.pt

pt · 13.6 MB · SHA-256 69783c858b44…04ea · Hugging Face

Download file

legacy/kd_mllm/s1_siglip2_qwen3_4b/v1-20260320-102316/checkpoint-2181/global_step2181/bf16_zero_pp_rank_3_mp_rank_00_optim_states.pt

pt · 13.6 MB · SHA-256 d309d9450112…89a3 · Hugging Face

Download file

legacy/kd_mllm/s1_siglip2_qwen3_4b/v1-20260320-102316/checkpoint-2181/global_step2181/bf16_zero_pp_rank_4_mp_rank_00_optim_states.pt

pt · 13.6 MB · SHA-256 764a29a55232…92d8 · Hugging Face

Download file

legacy/kd_mllm/s1_siglip2_qwen3_4b/v1-20260320-102316/checkpoint-2181/global_step2181/bf16_zero_pp_rank_5_mp_rank_00_optim_states.pt

pt · 13.6 MB · SHA-256 0219dcb18fe7…f004 · Hugging Face

Download file

legacy/kd_mllm/s1_siglip2_qwen3_4b/v1-20260320-102316/checkpoint-2181/global_step2181/bf16_zero_pp_rank_6_mp_rank_00_optim_states.pt

pt · 13.6 MB · SHA-256 44507259ca09…bb99 · Hugging Face

Download file

legacy/kd_mllm/s1_siglip2_qwen3_4b/v1-20260320-102316/checkpoint-2181/global_step2181/bf16_zero_pp_rank_7_mp_rank_00_optim_states.pt

pt · 13.6 MB · SHA-256 1abafa85e956…1ea6 · Hugging Face

Download file

legacy/kd_mllm/s1_siglip2_qwen3_4b/v1-20260320-102316/checkpoint-2181/global_step2181/mp_rank_00_model_states.pt

pt · 16.5 GB · SHA-256 3a3589c782b5…1dc0 · Hugging Face

Download file

legacy/kd_mllm/s1_siglip2_qwen3_4b/v1-20260320-102316/checkpoint-2181/model-00001-of-00002.safetensors

safetensors · 4.64 GB · SHA-256 8a36604a2184…33ec · Hugging Face

Download shard

legacy/kd_mllm/s1_siglip2_qwen3_4b/v1-20260320-102316/checkpoint-2181/model-00002-of-00002.safetensors

safetensors · 4.33 GB · SHA-256 69306dcccb26…bcae · Hugging Face

Download shard

legacy/kd_mllm/s2_siglip2_qwen3_4b_10pct/checkpoint-1000/global_step1000/bf16_zero_pp_rank_0_mp_rank_00_optim_states.pt

pt · 6.73 GB · SHA-256 2f8e8d79d11e…c9eb · Hugging Face

Download file

legacy/kd_mllm/s2_siglip2_qwen3_4b_10pct/checkpoint-1000/global_step1000/bf16_zero_pp_rank_1_mp_rank_00_optim_states.pt

pt · 6.73 GB · SHA-256 22bfae7b4846…0fa4 · Hugging Face

Download file

legacy/kd_mllm/s2_siglip2_qwen3_4b_10pct/checkpoint-1000/global_step1000/bf16_zero_pp_rank_2_mp_rank_00_optim_states.pt

pt · 6.73 GB · SHA-256 68a68d41b77b…f316 · Hugging Face

Download file

legacy/kd_mllm/s2_siglip2_qwen3_4b_10pct/checkpoint-1000/global_step1000/bf16_zero_pp_rank_3_mp_rank_00_optim_states.pt

pt · 6.73 GB · SHA-256 2c60aa9e9054…40f9 · Hugging Face

Download file

legacy/kd_mllm/s2_siglip2_qwen3_4b_10pct/checkpoint-1000/global_step1000/bf16_zero_pp_rank_4_mp_rank_00_optim_states.pt

pt · 6.73 GB · SHA-256 fd384be29c13…e27b · Hugging Face

Download file

legacy/kd_mllm/s2_siglip2_qwen3_4b_10pct/checkpoint-1000/global_step1000/bf16_zero_pp_rank_5_mp_rank_00_optim_states.pt

pt · 6.73 GB · SHA-256 698ec8800e65…6c29 · Hugging Face

Download file

legacy/kd_mllm/s2_siglip2_qwen3_4b_10pct/checkpoint-1000/global_step1000/bf16_zero_pp_rank_6_mp_rank_00_optim_states.pt

pt · 6.73 GB · SHA-256 0cf05b3bef5a…1472 · Hugging Face

Download file

legacy/kd_mllm/s2_siglip2_qwen3_4b_10pct/checkpoint-1000/global_step1000/bf16_zero_pp_rank_7_mp_rank_00_optim_states.pt

pt · 6.73 GB · SHA-256 6e2e79478dae…7b9d · Hugging Face

Download file

legacy/kd_mllm/s2_siglip2_qwen3_4b_10pct/checkpoint-1000/global_step1000/mp_rank_00_model_states.pt

pt · 8.98 GB · SHA-256 fbcaaf212e53…6280 · Hugging Face

Download file

legacy/kd_mllm/s2_siglip2_qwen3_4b_10pct/checkpoint-1000/model-00001-of-00002.safetensors

safetensors · 4.64 GB · SHA-256 b65f97a1d769…e188 · Hugging Face

Download shard

legacy/kd_mllm/s2_siglip2_qwen3_4b_10pct/checkpoint-1000/model-00002-of-00002.safetensors

safetensors · 4.33 GB · SHA-256 0ab5f6986d44…f56a · Hugging Face

Download shard

legacy/self_refine/qwen3vl_2b_10pct/checkpoint-1000/global_step1000/bf16_zero_pp_rank_0_mp_rank_00_optim_states.pt

pt · 2.40 GB · SHA-256 2a692536648e…3053 · Hugging Face

Download file

legacy/self_refine/qwen3vl_2b_10pct/checkpoint-1000/global_step1000/bf16_zero_pp_rank_1_mp_rank_00_optim_states.pt

pt · 2.40 GB · SHA-256 0e7522547b4c…d57a · Hugging Face

Download file

Source excerpts

3 excerpts

ReadmeJul 11huggingface.co/xiaomoguhzz/VisionEncoder

--- license: apache-2.0 library_name: transformers tags: - vision-encoder - distillation - video-language - siglip2 - dinov3 --- # VisionEncoder Hosted artifacts (derived data + trained checkpoints) for the **VisionEncoder** research project. **Training code + full reproduction guide**: https://github.com/xiaomoguhz/VisionEncoder The repo is organized into three top-level folders. ## `data/` — current (V9.x) reproduction data (~6.5G) | Path | Content | |---|---| | `data/vmllm_cached/qwen3vit/` | S2 `cached_dataset` arrow (image/video, 10pct + full); fed directly to stage-2 | | `data/ms-swift-data/` | sampled sharegpt jsonl (10pct + full) | | `data/llava_video/` | V9 decode-probed `good_manifest` for the video path | ## `ckpts/` — ready-made 4B MLLM inference weights | Path | Content | |---|---| | `ckpts/4b_stock` | 4B stock baseline (raw Qwen3.5 ViT, skips declip), checkpoint-505, 9.5G | | `ckpts/4b_v9_1` | 4B V9.1 (V-JEPA 2.1 video self-distill), checkpoint-505, 9.5G | Download either and feed it straight to evaluation (see the GitHub README, section 4 — MLLM evaluation) to skip declip + S1 + S2. ## `legacy/` — historical assets (~368G) Early-line products, not needed to reproduce the current main line: `declip_siglip2/spatial_align`, `kd_mllm`, `self_refine`, `video_mllm_swift` (old SigLIP2 / image-only S1+S2 ckpts), and old ViT-family arrow caches. ## Download ```bash # current dev data huggingface-cli download xiaomoguhzz/VisionEncoder --include "data/*" --local-dir . # ready-made 4B MLLM ckpt (eval directly) huggingface-cli download xiaomoguhzz/VisionEncoder --include "ckpts/4b_v9_1/*" --local-dir . ``` ## Related - Code + reproduction guide: https://github.com/xiaomoguhz/VisionEncoder

NoteJul 11huggingface.co/xiaomoguhzz/VisionEncoder

Source context: 0 downloads · 0 likes · Library transformers · Repo xiaomoguhzz/VisionEncoder

NoteJun 5huggingface.co/api/models/xiaomoguhzz/VisionEncoder

Source context: 0 downloads · 0 likes · Library transformers · Repo xiaomoguhzz/VisionEncoder

VisionEncoder

VisionEncoder

data/ — current (V9.x) reproduction data (~6.5G)

ckpts/ — ready-made 4B MLLM inference weights

legacy/ — historical assets (~368G)

Download

Related

Download files

`data/` — current (V9.x) reproduction data (~6.5G)

`ckpts/` — ready-made 4B MLLM inference weights

`legacy/` — historical assets (~368G)