ModelListedExternal

ZwZ 4B GGUF

These are quantizations of the model ZwZ-4B, using a imatrix created from text\en\medium

Sourcehuggingface.co/noctrex/ZwZ-4B-GGUF/tree/cd6da7f111a0f0dc68029fc13e9b819fb4809b9a

KindOtherBase modelinclusionAI/ZwZ-4BVersionRevision cd6da7f

Model source

Kind: Other
File size: 3.69 MB–8.22 GB
Base model: inclusionAI/ZwZ-4B
Version: vRevision cd6da7f
Updated: Sep 22, 2026
Downloads: 51
Tasks: Captioning
Source: Hugging Face

Source description

These are quantizations of the model ZwZ-4B, using a imatrix created from text_en_medium

Usage Notes:

Download the latest llama.cpp to use these quantizations.
Try to use the best quality you can run.
For the mmproj file, the F32 version is recommended for best results (F32 > BF16 > F16).

Sources

1 source

Hugging FacePrimaryVersion cd6da7f111a0f0dc68029fc13e9b819fb4809b9a

huggingface.co/noctrex/ZwZ-4B-GGUF/tree/cd6da7f111a0f0dc68029fc13e9b819fb4809b9a

Verified Sep 11

Model artifacts

11 artifacts

imatrix.gguf

gguf · 3.69 MB · SHA-256 70a45cd102ed…b18c · Hugging Face

Download

mmproj-BF16.gguf

gguf · 800 MB · SHA-256 4e800461385f…1074 · Hugging Face

Download

mmproj-F16.gguf

gguf · 797 MB · SHA-256 cf49ec6a15de…9642 · Hugging Face

Download

mmproj-F32.gguf

gguf · 1.55 GB · SHA-256 562f5c450a44…4a38 · Hugging Face

Download

ZwZ-4B-BF16.gguf

gguf · 8.22 GB · SHA-256 b83eb54ed7b6…1fe1 · Hugging Face

Download

ZwZ-4B-IQ4_NL.gguf

gguf · 2.42 GB · SHA-256 32011bbdad7a…5f1e · Hugging Face

Download

ZwZ-4B-IQ4_XS.gguf

gguf · 2.31 GB · SHA-256 e83fc278f5fe…55a0 · Hugging Face

Download

ZwZ-4B-Q4_K_M.gguf

gguf · 2.53 GB · SHA-256 29859b55352c…1a03 · Hugging Face

Download

ZwZ-4B-Q5_K_M.gguf

gguf · 2.94 GB · SHA-256 3295d67e606e…678f · Hugging Face

Download

ZwZ-4B-Q6_K.gguf

gguf · 3.38 GB · SHA-256 2571dbb93194…c69b · Hugging Face

Download

ZwZ-4B-Q8_0.gguf

gguf · 4.37 GB · SHA-256 96959206990a…825c · Hugging Face

Download

Source excerpts

2 excerpts

ReadmeSep 11huggingface.co/noctrex/ZwZ-4B-GGUF

--- pipeline_tag: image-to-text base_model: - inclusionAI/ZwZ-4B --- These are quantizations of the model [ZwZ-4B](https://huggingface.co/inclusionAI/ZwZ-4B), using a imatrix created from [text\_en\_medium](https://huggingface.co/datasets/eaddario/imatrix-calibration/blob/main/text_en_medium.parquet) **Usage Notes:** - Download the latest [llama.cpp](https://github.com/ggml-org/llama.cpp) to use these quantizations. - Try to use the best quality you can run. - For the `mmproj` file, the F32 version is recommended for best results (F32 > BF16 > F16).

NoteSep 11huggingface.co/noctrex/ZwZ-4B-GGUF

Source context: 51 downloads · 0 likes · Pipeline image-to-text · Repo noctrex/ZwZ-4B-GGUF