Skip to content
ModelVerifiedExternal
Download

Kernel LoRA v0.6

This is a QLoRA fine-tuned version of Qwen2.5-7B-Instruct, specialized in Linux Kernel knowledge. The model was fine-tuned on an M1 Pro (32GB) using MLX, with training data distilled from Qwen-3.7-Max.

Sourcehuggingface.co/gaowanlong/kernel-lora-v0.6/tree/89a0f35a08bfc344f03fc58b0fb76451264a2a7d
KindLoRABase modelQwen3VersionRevision 89a0f35Publisher@gaowanlongCgrade

Model source

Kind
LoRA
File size
3.99 GB
Base model
Qwen3
Version
vRevision 89a0f35
Updated
Oct 3, 2026
Publisher
@gaowanlong
Downloads
2
Source
Hugging Face

Source description

Kernel LoRA v0.6 — Knowledge Distillation from Qwen-3.7-Max

This is a QLoRA fine-tuned version of Qwen2.5-7B-Instruct, specialized in Linux Kernel knowledge. The model was fine-tuned on an M1 Pro (32GB) using MLX, with training data distilled from Qwen-3.7-Max.

Training Details

  • Base model: Qwen2.5-7B-Instruct (4-bit quantized)
  • Method: QLoRA (rank=8, scale=2.0, dropout=0.1)
  • Training data: 5,000 kernel Q&A samples distilled from Qwen-3.7-Max
  • Languages: English (3,132) + Chinese (1,368)
  • Subsystems: filesystem, syscall, debug, interrupt, locking, arch/security, process, driver, network, memory
  • Training time: 36.5 minutes on M1 Pro
  • Peak memory: 7.1 GB
  • Best val loss: 1.452 (step 39/200)

Evaluation

On a 39-question Linux Kernel knowledge test (LLM-as-judge scoring):

  • Base model: 74.1%
  • Fine-tuned: 69.2%
  • Delta: -4.9%

Best categories: Basic Concepts (+3.7%), Chinese Knowledge (-1.7%), Kernel Mechanisms (-3.7%)

Usage

from mlx_lm import load, generate

model, tokenizer = load("gaowanlong/kernel-lora-v0.6")

response = generate(
    model, tokenizer,
    prompt="What is the Linux kernel? Explain its role in an operating system.",
    max_tokens=300,
)
print(response)

Training History

VersionDataOverall DeltaBest Category
v0.1Raw kernel source-16.7%-
v0.2gzb666 + kernel source+2.8%Code Completion
v0.3gzb666 QA format-7.4%Code Completion
v0.4Eval-aligned data-5.1%Basic Concepts
v0.5Ewedubs commits-4.1%Advanced Internals
v0.6Qwen-3.7-Max distilled-4.9%Basic Concepts +3.7%

Limitations

  • Still experimental — knowledge improvements are modest
  • Best suited for kernel concept Q&A
  • Some degradation in code understanding tasks

Sources

1 source
Hugging FacePrimaryVersion 89a0f35a08bfc344f03fc58b0fb76451264a2a7d
huggingface.co/gaowanlong/kernel-lora-v0.6/tree/89a0f35a08bfc344f03fc58b0fb76451264a2a7d

Verified Oct 3

Model artifacts

1 artifact

model.safetensors

safetensors · 3.99 GB · SHA-256 a9289071e362…8dab · Hugging Face

Download file

Source excerpts

3 excerpts
ReadmeJul 11huggingface.co/gaowanlong/kernel-lora-v0.6

--- license: apache-2.0 language: - en - zh tags: - linux-kernel - qwen2.5 - lora - mlx - fine-tuning - knowledge-distillation datasets: - ewedubs/linux-kernel-commits-aireason-instruct library_name: mlx --- # Kernel LoRA v0.6 — Knowledge Distillation from Qwen-3.7-Max This is a QLoRA fine-tuned version of Qwen2.5-7B-Instruct, specialized in Linux Kernel knowledge. The model was fine-tuned on an M1 Pro (32GB) using MLX, with training data distilled from Qwen-3.7-Max. ## Training Details - **Base model**: Qwen2.5-7B-Instruct (4-bit quantized) - **Method**: QLoRA (rank=8, scale=2.0, dropout=0.1) - **Training data**: 5,000 kernel Q&A samples distilled from Qwen-3.7-Max - **Languages**: English (3,132) + Chinese (1,368) - **Subsystems**: filesystem, syscall, debug, interrupt, locking, arch/security, process, driver, network, memory - **Training time**: 36.5 minutes on M1 Pro - **Peak memory**: 7.1 GB - **Best val loss**: 1.452 (step 39/200) ## Evaluation On a 39-question Linux Kernel knowledge test (LLM-as-judge scoring): - Base model: 74.1% - Fine-tuned: 69.2% - Delta: -4.9% Best categories: Basic Concepts (+3.7%), Chinese Knowledge (-1.7%), Kernel Mechanisms (-3.7%) ## Usage ```python from mlx_lm import load, generate model, tokenizer = load("gaowanlong/kernel-lora-v0.6") response = generate( model, tokenizer, prompt="What is the Linux kernel? Explain its role in an operating system.", max_tokens=300, ) print(response) ``` ## Training History | Version | Data | Overall Delta | Best Category | |---------|------|--------------|---------------| | v0.1 | Raw kernel source | -16.7% | - | | v0.2 | gzb666 + kernel source | +2.8% | Code Completion | | v0.3 | gzb666 QA format | -7.4% | Code Completion | | v0.4 | Eval-aligned data | -5.1% | Basic Concepts | | v0.5 | Ewedubs commits | -4.1% | Advanced Internals | | **v0.6** | **Qwen-3.7-Max distilled** | **-4.9%** | **Basic Concepts +3.7%** | ## Limitations - Still experimental — knowledge improvements are modest - Best suited for kernel concept Q&A - Some degradation in code understanding tasks

NoteOct 3huggingface.co/gaowanlong/kernel-lora-v0.6

Source context: 2 downloads · 0 likes · Library mlx · Repo gaowanlong/kernel-lora-v0.6

NoteJul 17huggingface.co/gaowanlong/kernel-lora-v0.6

Source context: 35 downloads · 0 likes · Library mlx · Repo gaowanlong/kernel-lora-v0.6