ModelListedExternal

VAE Lyra XL Adaptive Cantor Illustrious

Multi-modal VAE trained with custom CLIP weights.

Sourcehuggingface.co/AbstractPhil/vae-lyra-xl-adaptive-cantor-illustrious/tree/3cba3dfdcd3e051a4eb5fcd74b5418a7e1d4fa09

KindOtherVersionRevision 3cba3dfBgrade

Model source

Kind: Other
File size: 913 MB–6.46 GB
Version: vRevision 3cba3df
Updated: Oct 9, 2026
Downloads: 190
Source: Hugging Face

Source description

VAE Lyra 🎵 - Illustrious Edition

Multi-modal VAE trained with custom CLIP weights.

CLIP Encoders

Uses CLIP weights from AbstractPhil/clips:

CLIP-L: IllustriousV01_clip_l.safetensors
CLIP-G: IllustriousV01_clip_g.safetensors

CLIP Skip: 2 (penultimate layer)

Model Details

Fusion Strategy: adaptive_cantor
Latent Dimension: 2048
Training Steps: 12,125
Best Loss: 0.0377
Prompt Source: booru

Quick Load (Safetensors)

from safetensors.torch import load_file

# Load just the weights (fast)
state_dict = load_file("weights/lyra_illustrious_best.safetensors")

# Or specific step
state_dict = load_file("weights/lyra_illustrious_step_5000.safetensors")

T5 Input Format

T5 receives a different input than CLIP to enable richer semantic understanding:

CLIP sees:  "masterpiece, 1girl, blue hair, school uniform, smile"
T5 sees:    "masterpiece, 1girl, blue hair, school uniform, smile ¶ A cheerful schoolgirl with blue hair smiling warmly"

The pilcrow (¶) separator acts as a mode-switch token.

Learned Parameters

Alpha (Visibility):

clip_g: 0.7316
clip_l: 0.7316
t5_xl_g: 0.7339
t5_xl_l: 0.7451

Beta (Capacity):

clip_l_t5_xl_l: 0.5709
clip_g_t5_xl_g: 0.5763

Usage

from lyra_xl_multimodal import load_lyra_from_hub

model = load_lyra_from_hub("AbstractPhil/vae-lyra-xl-adaptive-cantor-illustrious")
model.eval()

inputs = {
    "clip_l": clip_l_embeddings,     # [batch, 77, 768]
    "clip_g": clip_g_embeddings,     # [batch, 77, 1280]
    "t5_xl_l": t5_xl_embeddings,     # [batch, 512, 2048]
    "t5_xl_g": t5_xl_embeddings      # [batch, 512, 2048]
}

recons, mu, logvar, _ = model(inputs, target_modalities=["clip_l", "clip_g"])

Files

model.pt - Full checkpoint (model + optimizer + scheduler)
checkpoint_lyra_illustrious_XXXX.pt - Step checkpoints
config.json - Training configuration
weights/lyra_illustrious_best.safetensors - Best model weights only
weights/lyra_illustrious_step_XXXX.safetensors - Step checkpoints (weights only)

Sources

1 source

Hugging FacePrimaryVersion 3cba3dfdcd3e051a4eb5fcd74b5418a7e1d4fa09

huggingface.co/AbstractPhil/vae-lyra-xl-adaptive-cantor-illustrious/tree/3cba3dfdcd3e051a4eb5fcd74b5418a7e1d4fa09

Verified Oct 9

Download files

Model artifacts

64 artifacts

best_model.pt

pt · 2.55 GB · SHA-256 8778b72f1ce6…38eb · Hugging Face

Download file

checkpoint_1000.pt

pt · 2.55 GB · SHA-256 3c293c12f07d…620a · Hugging Face

Download file

checkpoint_2000_lyra_illustrious.pt

pt · 2.59 GB · SHA-256 f482e702c527…5b4f · Hugging Face

Download file

checkpoint_3000_lyra_illustrious.pt

pt · 2.59 GB · SHA-256 a3037d572b70…61f6 · Hugging Face

Download file

checkpoint_lyra_illustrious_10000.pt

pt · 2.58 GB · SHA-256 4a0c64b27be8…9624 · Hugging Face

Download file

checkpoint_lyra_illustrious_11000.pt

pt · 2.58 GB · SHA-256 aa1b58d4df70…9c5e · Hugging Face

Download file

checkpoint_lyra_illustrious_12000.pt

pt · 2.58 GB · SHA-256 56ec3fb0050b…8fc8 · Hugging Face

Download file

checkpoint_lyra_illustrious_13000.pt

pt · 2.58 GB · SHA-256 9f2586f510ef…3335 · Hugging Face

Download file

checkpoint_lyra_illustrious_14000.pt

pt · 2.58 GB · SHA-256 c121f1e62c52…b69f · Hugging Face

Download file

checkpoint_lyra_illustrious_15000.pt

pt · 2.58 GB · SHA-256 559139a15874…ebf3 · Hugging Face

Download file

checkpoint_lyra_illustrious_16000.pt

pt · 2.58 GB · SHA-256 d18ced48fe47…3e37 · Hugging Face

Download file

checkpoint_lyra_illustrious_17000.pt

pt · 2.58 GB · SHA-256 34a00e5d2c71…5257 · Hugging Face

Download file

checkpoint_lyra_illustrious_18000.pt

pt · 2.58 GB · SHA-256 05eb58f065e4…47e0 · Hugging Face

Download file

checkpoint_lyra_illustrious_19000.pt

pt · 2.58 GB · SHA-256 81c17327c43f…db28 · Hugging Face

Download file

checkpoint_lyra_illustrious_20000.pt

pt · 2.58 GB · SHA-256 1e0e322e1862…bc4f · Hugging Face

Download file

checkpoint_lyra_illustrious_21000.pt

pt · 2.58 GB · SHA-256 9e444c33d6d2…760b · Hugging Face

Download file

checkpoint_lyra_illustrious_22000.pt

pt · 2.58 GB · SHA-256 d549f7ae3863…3b81 · Hugging Face

Download file

checkpoint_lyra_illustrious_23000.pt

pt · 2.58 GB · SHA-256 1be74662c7b0…3b18 · Hugging Face

Download file

checkpoint_lyra_illustrious_24000.pt

pt · 2.58 GB · SHA-256 7739e82a17db…4a99 · Hugging Face

Download file

checkpoint_lyra_illustrious_25000.pt

pt · 2.58 GB · SHA-256 9f5a4fcf3a09…5f4b · Hugging Face

Download file

checkpoint_lyra_illustrious_26000.pt

pt · 2.58 GB · SHA-256 ac41efbdf58c…205e · Hugging Face

Download file

checkpoint_lyra_illustrious_27000.pt

pt · 2.58 GB · SHA-256 b5c25ee486f3…6c5a · Hugging Face

Download file

checkpoint_lyra_illustrious_28000.pt

pt · 2.58 GB · SHA-256 d329624b634e…a7ef · Hugging Face

Download file

checkpoint_lyra_illustrious_29000.pt

pt · 2.58 GB · SHA-256 c2cef2b70e01…1e87 · Hugging Face

Download file

checkpoint_lyra_illustrious_30000.pt

pt · 2.58 GB · SHA-256 86c942ae97a9…05ae · Hugging Face

Download file

checkpoint_lyra_illustrious_31000.pt

pt · 2.58 GB · SHA-256 873cb95e459a…c2f7 · Hugging Face

Download file

checkpoint_lyra_illustrious_32000.pt

pt · 2.58 GB · SHA-256 372b974d8942…e562 · Hugging Face

Download file

checkpoint_lyra_illustrious_33000.pt

pt · 2.58 GB · SHA-256 be1a2eab6dcb…ad13 · Hugging Face

Download file

checkpoint_lyra_illustrious_34000.pt

pt · 2.58 GB · SHA-256 733ac1455fbc…4dd7 · Hugging Face

Download file

checkpoint_lyra_illustrious_35000.pt

pt · 2.58 GB · SHA-256 0ea1ed5b2d84…9b5e · Hugging Face

Download file

checkpoint_lyra_illustrious_36000.pt

pt · 2.58 GB · SHA-256 88c3b5ad8fb1…1f10 · Hugging Face

Download file

checkpoint_lyra_illustrious_37000.pt

pt · 2.58 GB · SHA-256 935f4b9d5546…a951 · Hugging Face

Download file

checkpoint_lyra_illustrious_38000.pt

pt · 2.58 GB · SHA-256 e11ee279ce16…21f3 · Hugging Face

Download file

checkpoint_lyra_illustrious_39000.pt

pt · 2.58 GB · SHA-256 b8ceb4759500…3132 · Hugging Face

Download file

checkpoint_lyra_illustrious_4000.pt

pt · 2.58 GB · SHA-256 28196bc43bdf…3fd3 · Hugging Face

Download file

checkpoint_lyra_illustrious_40000.pt

pt · 2.58 GB · SHA-256 2464e6d92116…a40d · Hugging Face

Download file

checkpoint_lyra_illustrious_41000.pt

pt · 2.58 GB · SHA-256 9812ff84c4dc…e40a · Hugging Face

Download file

checkpoint_lyra_illustrious_5000.pt

pt · 2.58 GB · SHA-256 1cb87a8919ff…1376 · Hugging Face

Download file

checkpoint_lyra_illustrious_6000.pt

pt · 2.58 GB · SHA-256 f8cf28181fab…7a91 · Hugging Face

Download file

checkpoint_lyra_illustrious_7000.pt

pt · 2.58 GB · SHA-256 ed301ae8a38b…c25b · Hugging Face

Download file

checkpoint_lyra_illustrious_8000.pt

pt · 2.58 GB · SHA-256 0bc0d3e32f98…1393 · Hugging Face

Download file

checkpoint_lyra_illustrious_9000.pt

pt · 2.58 GB · SHA-256 b26ba85f69a6…01a5 · Hugging Face

Download file

illustriousXL_v01.safetensors

safetensors · 6.46 GB · SHA-256 3e15ba00387d…5aa9 · Hugging Face

Download file

model.pt

pt · 2.58 GB · SHA-256 4433cb800b6e…3846 · Hugging Face

Download file

weights/lyra_illustrious_best.safetensors

safetensors · 913 MB · SHA-256 3c4bf295b52c…631d · Hugging Face

Download file

weights/lyra_illustrious_step_10000.safetensors

safetensors · 913 MB · SHA-256 188121373ba6…9bb1 · Hugging Face

Download file

weights/lyra_illustrious_step_11000.safetensors

safetensors · 913 MB · SHA-256 64eaae2334d1…337c · Hugging Face

Download file

weights/lyra_illustrious_step_12000.safetensors

safetensors · 913 MB · SHA-256 ab2841ef2b50…189b · Hugging Face

Download file

weights/lyra_illustrious_step_13000.safetensors

safetensors · 913 MB · SHA-256 8d744ab64651…0ea7 · Hugging Face

Download file

weights/lyra_illustrious_step_14000.safetensors

safetensors · 913 MB · SHA-256 b8a83950c0de…2b6e · Hugging Face

Download file

weights/lyra_illustrious_step_15000.safetensors

safetensors · 913 MB · SHA-256 ec7becbada8a…cb4a · Hugging Face

Download file

weights/lyra_illustrious_step_16000.safetensors

safetensors · 913 MB · SHA-256 c8a4cc9c420d…06ea · Hugging Face

Download file

weights/lyra_illustrious_step_17000.safetensors

safetensors · 913 MB · SHA-256 c0b79191c4c4…26f9 · Hugging Face

Download file

weights/lyra_illustrious_step_18000.safetensors

safetensors · 913 MB · SHA-256 3aabee6d312e…5ea8 · Hugging Face

Download file

weights/lyra_illustrious_step_19000.safetensors

safetensors · 913 MB · SHA-256 74c82b30c835…9ba6 · Hugging Face

Download file

weights/lyra_illustrious_step_20000.safetensors

safetensors · 913 MB · SHA-256 f92cadd3e83d…233d · Hugging Face

Download file

weights/lyra_illustrious_step_21000.safetensors

safetensors · 913 MB · SHA-256 2cee8a2111ef…0f15 · Hugging Face

Download file

weights/lyra_illustrious_step_22000.safetensors

safetensors · 913 MB · SHA-256 dd524069d8f8…cdb9 · Hugging Face

Download file

weights/lyra_illustrious_step_23000.safetensors

safetensors · 913 MB · SHA-256 ddb59a637a96…9f44 · Hugging Face

Download file

weights/lyra_illustrious_step_24000.safetensors

safetensors · 913 MB · SHA-256 f619118f7c97…d186 · Hugging Face

Download file

weights/lyra_illustrious_step_25000.safetensors

safetensors · 913 MB · SHA-256 c1d63fa0ac84…60cc · Hugging Face

Download file

weights/lyra_illustrious_step_26000.safetensors

safetensors · 913 MB · SHA-256 27a9541f0343…3da9 · Hugging Face

Download file

weights/lyra_illustrious_step_27000.safetensors

safetensors · 913 MB · SHA-256 af5adb707c94…c44a · Hugging Face

Download file

weights/lyra_illustrious_step_28000.safetensors

safetensors · 913 MB · SHA-256 f39821c978ea…0fbf · Hugging Face

Download file

Source excerpts

3 excerpts

ReadmeJul 31huggingface.co/AbstractPhil/vae-lyra-xl-adaptive-cantor-illustrious

--- tags: - vae - multimodal - clip - t5 - sdxl - illustrious - adaptive-cantor - booru license: mit --- # VAE Lyra 🎵 - Illustrious Edition Multi-modal VAE trained with **custom CLIP weights**. ## CLIP Encoders Uses CLIP weights from `AbstractPhil/clips`: - CLIP-L: `IllustriousV01_clip_l.safetensors` - CLIP-G: `IllustriousV01_clip_g.safetensors` **CLIP Skip**: 2 (penultimate layer) ## Model Details - **Fusion Strategy**: adaptive_cantor - **Latent Dimension**: 2048 - **Training Steps**: 12,125 - **Best Loss**: 0.0377 - **Prompt Source**: booru ## Quick Load (Safetensors) ```python from safetensors.torch import load_file # Load just the weights (fast) state_dict = load_file("weights/lyra_illustrious_best.safetensors") # Or specific step state_dict = load_file("weights/lyra_illustrious_step_5000.safetensors") ``` ## T5 Input Format T5 receives a different input than CLIP to enable richer semantic understanding: ``` CLIP sees: "masterpiece, 1girl, blue hair, school uniform, smile" T5 sees: "masterpiece, 1girl, blue hair, school uniform, smile ¶ A cheerful schoolgirl with blue hair smiling warmly" ``` The pilcrow (`¶`) separator acts as a mode-switch token. ## Learned Parameters **Alpha (Visibility):** - clip_g: 0.7316 - clip_l: 0.7316 - t5_xl_g: 0.7339 - t5_xl_l: 0.7451 **Beta (Capacity):** - clip_l_t5_xl_l: 0.5709 - clip_g_t5_xl_g: 0.5763 ## Usage ```python from lyra_xl_multimodal import load_lyra_from_hub model = load_lyra_from_hub("AbstractPhil/vae-lyra-xl-adaptive-cantor-illustrious") model.eval() inputs = { "clip_l": clip_l_embeddings, # [batch, 77, 768] "clip_g": clip_g_embeddings, # [batch, 77, 1280] "t5_xl_l": t5_xl_embeddings, # [batch, 512, 2048] "t5_xl_g": t5_xl_embeddings # [batch, 512, 2048] } recons, mu, logvar, _ = model(inputs, target_modalities=["clip_l", "clip_g"]) ``` ## Files - `model.pt` - Full checkpoint (model + optimizer + scheduler) - `checkpoint_lyra_illustrious_XXXX.pt` - Step checkpoints - `config.json` - Training configuration - `weights/lyra_illustrious_best.safetensors` - Best model weights only - `weights/lyra_illustrious_step_XXXX.safetensors` - Step checkpoints (weights only)

NoteOct 9huggingface.co/AbstractPhil/vae-lyra-xl-adaptive-cantor-illustrious

Source context: 190 downloads · 1 likes · Repo AbstractPhil/vae-lyra-xl-adaptive-cantor-illustrious

NoteJul 31huggingface.co/AbstractPhil/vae-lyra-xl-adaptive-cantor-illustrious

Source context: 37 downloads · 1 likes · Repo AbstractPhil/vae-lyra-xl-adaptive-cantor-illustrious