ModelListedExternal

ZwZ 8B GGUF

These are quantizations of the model ZwZ-8B, using a imatrix created from text\en\medium

Sourcehuggingface.co/noctrex/ZwZ-8B-GGUF/tree/fc6cca531f9ccd3ccff55cdf3c685a773f3f3f4a

KindOtherBase modelinclusionAI/ZwZ-8BVersionRevision fc6cca5

Model source

Kind: Other
File size: 5.10 MB–15.3 GB
Base model: inclusionAI/ZwZ-8B
Version: vRevision fc6cca5
Updated: Sep 25, 2026
Downloads: 252
Tasks: Captioning
Source: Hugging Face

Source description

These are quantizations of the model ZwZ-8B, using a imatrix created from text_en_medium

Usage Notes:

Download the latest llama.cpp to use these quantizations.
Try to use the best quality you can run.
For the mmproj file, the F32 version is recommended for best results (F32 > BF16 > F16).

Sources

1 source

Hugging FacePrimaryVersion fc6cca531f9ccd3ccff55cdf3c685a773f3f3f4a

huggingface.co/noctrex/ZwZ-8B-GGUF/tree/fc6cca531f9ccd3ccff55cdf3c685a773f3f3f4a

Verified Sep 11

Model artifacts

11 artifacts

imatrix.gguf

gguf · 5.10 MB · SHA-256 9cd2572aa733…b427 · Hugging Face

Download

mmproj-BF16.gguf

gguf · 1.08 GB · SHA-256 e592c7557b72…6b6c · Hugging Face

Download

mmproj-F16.gguf

gguf · 1.08 GB · SHA-256 52853561b1fe…66b8 · Hugging Face

Download

mmproj-F32.gguf

gguf · 2.15 GB · SHA-256 a2d991a4b755…5634 · Hugging Face

Download

ZwZ-8B-BF16.gguf

gguf · 15.3 GB · SHA-256 5d7e72b905b1…1728 · Hugging Face

Download

ZwZ-8B-IQ4_NL.gguf

gguf · 4.46 GB · SHA-256 6881e41e3184…b230 · Hugging Face

Download

ZwZ-8B-IQ4_XS.gguf

gguf · 4.25 GB · SHA-256 9ab7a2542c9f…211d · Hugging Face

Download

ZwZ-8B-Q4_K_M.gguf

gguf · 4.68 GB · SHA-256 f39873f8a587…a93c · Hugging Face

Download

ZwZ-8B-Q5_K_M.gguf

gguf · 5.45 GB · SHA-256 6fb65cc36d0f…5c0b · Hugging Face

Download

ZwZ-8B-Q6_K.gguf

gguf · 6.26 GB · SHA-256 da6593e517a4…68e9 · Hugging Face

Download

ZwZ-8B-Q8_0.gguf

gguf · 8.11 GB · SHA-256 4b422b58d8c1…1856 · Hugging Face

Download

Source excerpts

2 excerpts

ReadmeSep 11huggingface.co/noctrex/ZwZ-8B-GGUF

--- pipeline_tag: image-to-text base_model: - inclusionAI/ZwZ-8B --- These are quantizations of the model [ZwZ-8B](https://huggingface.co/inclusionAI/ZwZ-8B), using a imatrix created from [text\_en\_medium](https://huggingface.co/datasets/eaddario/imatrix-calibration/blob/main/text_en_medium.parquet) **Usage Notes:** - Download the latest [llama.cpp](https://github.com/ggml-org/llama.cpp) to use these quantizations. - Try to use the best quality you can run. - For the `mmproj` file, the F32 version is recommended for best results (F32 > BF16 > F16).

NoteSep 11huggingface.co/noctrex/ZwZ-8B-GGUF

Source context: 252 downloads · 4 likes · Pipeline image-to-text · Repo noctrex/ZwZ-8B-GGUF