Last Vit Clip Vitb16

--- tags: - vision - vit - last-vit - clip - text-supervised license: mit library_name: transformers --- # LAST-ViT CLIP ViT-B/16 (Text-Supervised) Vision encoder from [OpenCLIP](https://github.com/mlfoundations/open_clip) ViT-B/16 with LAST-ViT pooling. Converted to HuggingFace Transformers format from the [LAST-ViT](https://github.com/ChengShiest/LAST-ViT) openai_b_16 checkpoint. This is the **vision encoder only** (no text encoder). Use for image embedding extraction or as a backbone for vision tasks. ## Model Details - **Architecture:** ViT-B/16 (Vision Transformer Base, patch size 16×16) - **Training scenario:** Text-supervised (CLIP) - **Hidden size:** 768 - **Num layers:** 12 - **Num attention heads:** 12 - **Image size:** 224×224 ## Usage ```python from transformers import ViTModel, ViTImageProcessor from PIL import Image model = ViTModel.from_pretrained("BiliSakura/last-vit-clip-vitb16") processor = ViTImageProcessor.from_pretrained("BiliSakura/last-vit-clip-vitb16") image = Image.open("your_image.jpg").convert("RGB") inputs = processor(images=image, return_tensors="pt") outputs = model(**inputs) # Image embeddings (CLS token) embeddings = outputs.last_hidden_state[:, 0, :] # (batch, 768) ``` ## References - **Repository:** [ChengShiest/LAST-ViT](https://github.com/ChengShiest/LAST-ViT) - **CLIP training:** [mlfoundations/open_clip](https://github.com/mlfoundations/open_clip) - **Original weights:** [openai_b_16.pt](https://github.com/ChengShiest/LAST-ViT/releases/download/weights/openai_b_16.pt)

Source context: 17 downloads · 0 likes · Pipeline image-feature-extraction · Library transformers · Repo BiliSakura/last-vit-clip-vitb16